2022-04-26T06:03:32.829Z [main] WARN o.o.s.c.util.YamlToObjectConverter RequestID=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= The file /app/jetty/config/catalog-be/neo4j-errors-configuration.yaml cannot be found. Ignore reading configuration. 2022-04-26T06:03:32.958Z [main] INFO o.o.s.be.listen.BEAppContextListener RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ASDC version from MANIFEST is 1.10.4 2022-04-26T06:03:33.027Z [main] INFO o.o.s.b.m.BeMonitoringService RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= BE monitoring service is disabled 2022-04-26T06:03:33.029Z [main] INFO o.s.web.context.ContextLoader RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Root WebApplicationContext: initialization started 2022-04-26T06:03:35.647Z [main] INFO o.o.s.b.d.j.JanusGraphClient RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ** JanusGraphClient created 2022-04-26T06:03:35.652Z [main] INFO o.o.s.b.d.j.JanusGraphClient RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ** createGraph started ** 2022-04-26T06:03:35.652Z [main] INFO o.o.s.b.d.j.JanusGraphClient RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ** open graph with /app/jetty/config/catalog-be/janusgraph.properties started 2022-04-26T06:03:35.652Z [main] INFO o.o.s.b.d.j.JanusGraphClient RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= openGraph : try to load file /app/jetty/config/catalog-be/janusgraph.properties 2022-04-26T06:03:36.121Z [main] INFO com.datastax.driver.core RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= DataStax Java driver 3.8.0 for Apache Cassandra 2022-04-26T06:03:36.127Z [main] INFO c.d.driver.core.GuavaCompatibility RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Detected Guava >= 19 in the classpath, using modern compatibility layer 2022-04-26T06:03:36.153Z [main] INFO c.d.driver.core.SystemProperties RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= com.datastax.driver.USE_NATIVE_CLOCK is defined, using value false 2022-04-26T06:03:36.153Z [main] INFO c.datastax.driver.core.ClockFactory RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Using java.lang.System clock to generate timestamps. 2022-04-26T06:03:36.374Z [main] INFO com.datastax.driver.core.NettyUtil RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Found Netty's native epoll transport in the classpath, using it 2022-04-26T06:03:36.774Z [main] INFO o.o.s.b.d.j.JanusGraphClient RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= createGraph : failed to open JanusGraph graph with configuration file: /app/jetty/config/catalog-be/janusgraph.properties 2022-04-26T06:03:36.778Z [main] INFO o.o.s.b.d.j.JanusGraphClient RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Scheduling reconnect task org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$ReconnectTask@646ddc20 with interval of 3 seconds 2022-04-26T06:03:36.780Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient ** open graph with /app/jetty/config/catalog-be/janusgraph.properties started 2022-04-26T06:03:36.786Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient openGraph : try to load file /app/jetty/config/catalog-be/janusgraph.properties 2022-04-26T06:03:36.793Z [JanusGraph-Reconnect-Task] INFO c.d.driver.core.SystemProperties com.datastax.driver.USE_NATIVE_CLOCK is defined, using value false 2022-04-26T06:03:36.793Z [JanusGraph-Reconnect-Task] INFO c.datastax.driver.core.ClockFactory Using java.lang.System clock to generate timestamps. 2022-04-26T06:03:37.101Z [main] INFO o.o.s.b.d.j.JanusGraphGenericDao RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ** JanusGraphGenericDao created 2022-04-26T06:03:37.114Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient createGraph : failed to open JanusGraph graph with configuration file: /app/jetty/config/catalog-be/janusgraph.properties 2022-04-26T06:03:37.226Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= All sdc keyspaces are : [sdcartifact, sdcrepository, sdcaudit] 2022-04-26T06:03:37.227Z [main] INFO o.o.s.b.d.c.schema.SdcSchemaUtils RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Connecting to node: [cassandra.onap] port: 9042. 2022-04-26T06:03:37.229Z [main] INFO o.o.s.b.d.c.schema.SdcSchemaUtils RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Connection timeout in seconds : 120 2022-04-26T06:03:37.230Z [main] INFO o.o.s.b.d.c.schema.SdcSchemaUtils RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= SocketConnectTimeout was provided, setting Cassandra client to use SocketConnectTimeout: 20000 . 2022-04-26T06:03:37.230Z [main] INFO o.o.s.b.d.c.schema.SdcSchemaUtils RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= SocketReadTimeout was provided, setting Cassandra client to use SocketReadTimeout: 20000 . 2022-04-26T06:03:37.230Z [main] INFO o.o.s.b.d.c.schema.SdcSchemaUtils RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= localDatacenter was provided, setting Cassndra clint to use datacenter: Pod as local. 2022-04-26T06:03:37.231Z [main] INFO c.d.driver.core.SystemProperties RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= com.datastax.driver.USE_NATIVE_CLOCK is defined, using value false 2022-04-26T06:03:37.231Z [main] INFO c.datastax.driver.core.ClockFactory RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Using java.lang.System clock to generate timestamps. 2022-04-26T06:03:37.232Z [main] INFO o.o.s.b.d.c.schema.SdcSchemaUtils RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ** cluster created 2022-04-26T06:03:37.232Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= creating cluster for Cassandra Health Check. 2022-04-26T06:03:37.777Z [main] INFO c.d.d.c.p.DCAwareRoundRobinPolicy RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Using provided data-center name 'Pod' for DCAwareRoundRobinPolicy 2022-04-26T06:03:37.779Z [main] INFO com.datastax.driver.core.Cluster RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= New Cassandra host cassandra.onap/10.233.72.7:9042 added 2022-04-26T06:03:37.779Z [main] INFO com.datastax.driver.core.Cluster RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= New Cassandra host cassandra.onap/10.233.65.18:9042 added 2022-04-26T06:03:37.780Z [main] INFO com.datastax.driver.core.Cluster RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= New Cassandra host cassandra.onap/10.233.68.64:9042 added 2022-04-26T06:03:37.780Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Cluster Metadata: com.datastax.driver.core.Metadata@727de450 2022-04-26T06:03:37.781Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= keyspace : sdcartifact , replication: {Pod=3, class=org.apache.cassandra.locator.NetworkTopologyStrategy} 2022-04-26T06:03:37.781Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= keyspace : sdcrepository , replication: {Pod=3, class=org.apache.cassandra.locator.NetworkTopologyStrategy} 2022-04-26T06:03:37.781Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= keyspace : sdcaudit , replication: {Pod=3, class=org.apache.cassandra.locator.NetworkTopologyStrategy} 2022-04-26T06:03:37.782Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= maxReplication Factor is: 3 2022-04-26T06:03:37.782Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= localQuorum is: 2 2022-04-26T06:03:37.783Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Health Check formula : Replication Factor – Local_Quorum = 1 2022-04-26T06:03:37.800Z [main] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ECOMP Portal health check configuration is missing. 2022-04-26T06:03:37.808Z [main] INFO o.o.s.b.s.d.SwitchoverDetector RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Enter init method of SwitchoverDetector 2022-04-26T06:03:37.809Z [main] INFO o.o.s.b.s.d.SwitchoverDetector RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null switchover detector service is disabled 2022-04-26T06:03:37.835Z [cluster1-nio-worker-0] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.233.72.7:9042 com.datastax.driver.core.exceptions.TransportException: [cassandra.onap/10.233.72.7:9042] Cannot connect at com.datastax.driver.core.Connection$1.operationComplete(Connection.java:193) at com.datastax.driver.core.Connection$1.operationComplete(Connection.java:163) at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578) at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:571) at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:550) at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491) at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616) at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609) at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:470) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: cassandra.onap/10.233.72.7:9042 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at io.netty.channel.unix.Errors.newConnectException0(Errors.java:155) at io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128) at io.netty.channel.unix.Socket.finishConnect(Socket.java:320) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:470) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:03:37.836Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.233.72.7:9042 com.datastax.driver.core.exceptions.ConnectionException: [cassandra.onap/10.233.72.7:9042] Pool was closed during initialization at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:173) at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:158) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1080) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.google.common.util.concurrent.CollectionFuture.handleAllCompleted(CollectionFuture.java:69) at com.google.common.util.concurrent.AggregateFuture.processCompleted(AggregateFuture.java:282) at com.google.common.util.concurrent.AggregateFuture.decrementCountAndMaybeComplete(AggregateFuture.java:264) at com.google.common.util.concurrent.AggregateFuture.access$200(AggregateFuture.java:42) at com.google.common.util.concurrent.AggregateFuture$1.run(AggregateFuture.java:147) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:100) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:53) at com.datastax.driver.core.Connection$1.operationComplete(Connection.java:191) at com.datastax.driver.core.Connection$1.operationComplete(Connection.java:163) at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578) at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:571) at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:550) at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491) at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616) at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609) at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:470) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:03:37.984Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null creating cluster for Cassandra for monitoring. 2022-04-26T06:03:37.985Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null The cassandra session is com.datastax.driver.core.SessionManager@2cdb53d6 2022-04-26T06:03:37.985Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null The number of cassandra nodes is:3 2022-04-26T06:03:37.987Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null The cassandra down nodes number is 1 2022-04-26T06:03:39.194Z [main] INFO o.o.s.b.dao.janusgraph.JanusGraphDao RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ** JanusGraphDao created 2022-04-26T06:03:39.243Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=54121e21-06aa-418a-a0b7-1edf162058e9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:03:39.245Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=54121e21-06aa-418a-a0b7-1edf162058e9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@752f5ee7 2022-04-26T06:03:39.245Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=54121e21-06aa-418a-a0b7-1edf162058e9 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:03:39.246Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=54121e21-06aa-418a-a0b7-1edf162058e9 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:03:39.277Z [main] INFO o.o.s.b.d.cassandra.CassandraClient RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null SocketConnectTimeout was provided, setting Cassandra client to use SocketConnectTimeout: 20000 . 2022-04-26T06:03:39.281Z [main] INFO o.o.s.b.d.cassandra.CassandraClient RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null SocketReadTimeout was provided, setting Cassandra client to use SocketReadTimeout: 20000 . 2022-04-26T06:03:39.282Z [main] INFO o.o.s.b.d.cassandra.CassandraClient RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null localDatacenter was provided, setting Cassndra clint to use datacenter: Pod as local. 2022-04-26T06:03:39.282Z [main] INFO c.d.driver.core.SystemProperties RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null com.datastax.driver.USE_NATIVE_CLOCK is defined, using value false 2022-04-26T06:03:39.282Z [main] INFO c.datastax.driver.core.ClockFactory RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Using java.lang.System clock to generate timestamps. 2022-04-26T06:03:39.282Z [main] INFO o.o.s.b.d.cassandra.CassandraClient RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ** CassandraClient created 2022-04-26T06:03:39.665Z [main] INFO c.d.d.c.p.DCAwareRoundRobinPolicy RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Using provided data-center name 'Pod' for DCAwareRoundRobinPolicy 2022-04-26T06:03:39.667Z [main] INFO com.datastax.driver.core.Cluster RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null New Cassandra host cassandra.onap/10.233.65.18:9042 added 2022-04-26T06:03:39.667Z [main] INFO com.datastax.driver.core.Cluster RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null New Cassandra host cassandra.onap/10.233.68.64:9042 added 2022-04-26T06:03:39.779Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient ** open graph with /app/jetty/config/catalog-be/janusgraph.properties started 2022-04-26T06:03:39.780Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient openGraph : try to load file /app/jetty/config/catalog-be/janusgraph.properties 2022-04-26T06:03:39.784Z [JanusGraph-Reconnect-Task] INFO c.d.driver.core.SystemProperties com.datastax.driver.USE_NATIVE_CLOCK is defined, using value false 2022-04-26T06:03:39.784Z [JanusGraph-Reconnect-Task] INFO c.datastax.driver.core.ClockFactory Using java.lang.System clock to generate timestamps. 2022-04-26T06:03:39.796Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient createGraph : failed to open JanusGraph graph with configuration file: /app/jetty/config/catalog-be/janusgraph.properties 2022-04-26T06:03:39.902Z [main] INFO o.o.s.b.d.c.ToscaModelImportCassandraDao RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null org.openecomp.sdc.be.dao.cassandra.ToscaModelImportCassandraDao successfully initialized 2022-04-26T06:03:40.305Z [main] INFO o.o.s.b.d.c.AuditCassandraDao RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ** AuditCassandraDao created 2022-04-26T06:03:40.346Z [main] INFO o.o.s.b.c.d.e.DmaapClientFactory RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650952996, consumerId=sdc-AUTO1-1650952996, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:03:40.432Z [main] ERROR o.o.s.be.catalog.impl.DmaapProducer RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:03:40.881Z [main] INFO o.o.s.b.d.c.ArtifactCassandraDao RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ** ArtifactCassandraDao created 2022-04-26T06:03:40.929Z [main] ERROR o.o.s.b.m.o.impl.DataTypeOperation RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null UNKNOWN_ERROR 2022-04-26T06:03:40.945Z [main] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=200 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=org.openecomp.sdc.be.model.operations.impl.DataTypeOperation Internal Connection problem. Description: Failed to fetch data types from database with model null. Status is NOT_CREATED 2022-04-26T06:03:40.954Z [main] ERROR o.o.s.b.m.o.impl.ModelOperation RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=300 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null null 2022-04-26T06:03:40.958Z [main] WARN o.s.w.c.s.XmlWebApplicationContext RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=300 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'artifactServlet' defined in file [/tmp/jetty-0_0_0_0-8080-catalog-be_war-_-any-1713229416780379629/webapp/WEB-INF/classes/org/openecomp/sdc/be/servlets/ArtifactServlet.class]: Unsatisfied dependency expressed through constructor parameter 2; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'artifactBusinessLogic' defined in file [/tmp/jetty-0_0_0_0-8080-catalog-be_war-_-any-1713229416780379629/webapp/WEB-INF/classes/org/openecomp/sdc/be/components/impl/ArtifactsBusinessLogic.class]: Unsatisfied dependency expressed through constructor parameter 1; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'tosca-export-handler' defined in file [/tmp/jetty-0_0_0_0-8080-catalog-be_war-_-any-1713229416780379629/webapp/WEB-INF/classes/org/openecomp/sdc/be/tosca/ToscaExportHandler.class]: Unsatisfied dependency expressed through constructor parameter 3; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'policyExportParserImpl' defined in file [/tmp/jetty-0_0_0_0-8080-catalog-be_war-_-any-1713229416780379629/webapp/WEB-INF/classes/org/openecomp/sdc/be/tosca/PolicyExportParserImpl.class]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.openecomp.sdc.be.tosca.PolicyExportParserImpl]: Constructor threw exception; nested exception is org.openecomp.sdc.be.model.jsonjanusgraph.operations.exception.OperationException 2022-04-26T06:03:42.280Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02b7ffdf-535e-4855-a906-e15f0deec697 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:03:42.280Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02b7ffdf-535e-4855-a906-e15f0deec697 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@b25bc7e 2022-04-26T06:03:42.280Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02b7ffdf-535e-4855-a906-e15f0deec697 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:03:42.280Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02b7ffdf-535e-4855-a906-e15f0deec697 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:03:42.779Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient ** open graph with /app/jetty/config/catalog-be/janusgraph.properties started 2022-04-26T06:03:42.780Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient openGraph : try to load file /app/jetty/config/catalog-be/janusgraph.properties 2022-04-26T06:03:42.783Z [JanusGraph-Reconnect-Task] INFO c.d.driver.core.SystemProperties com.datastax.driver.USE_NATIVE_CLOCK is defined, using value false 2022-04-26T06:03:42.783Z [JanusGraph-Reconnect-Task] INFO c.datastax.driver.core.ClockFactory Using java.lang.System clock to generate timestamps. 2022-04-26T06:03:42.794Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient createGraph : failed to open JanusGraph graph with configuration file: /app/jetty/config/catalog-be/janusgraph.properties 2022-04-26T06:03:42.978Z [main] INFO o.o.s.b.d.cassandra.CassandraClient RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=300 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ** CassandraClient cluster closed 2022-04-26T06:03:44.990Z [main] INFO o.o.s.b.d.c.schema.SdcSchemaUtils RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=300 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ** CassandraClient cluster closed 2022-04-26T06:03:44.990Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=300 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ** sdcSchemaUtils cluster closed 2022-04-26T06:03:44.992Z [main] WARN o.s.c.a.CommonAnnotationBeanPostProcessor RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=300 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Destroy method on bean with name 'janusgraph-client' threw an exception: java.lang.NullPointerException 2022-04-26T06:03:45.003Z [main] ERROR o.s.web.context.ContextLoader RequestID=721783c9-6538-4bc8-b05c-7ed681bed444 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=300 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Context initialization failed org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'artifactServlet' defined in file [/tmp/jetty-0_0_0_0-8080-catalog-be_war-_-any-1713229416780379629/webapp/WEB-INF/classes/org/openecomp/sdc/be/servlets/ArtifactServlet.class]: Unsatisfied dependency expressed through constructor parameter 2; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'artifactBusinessLogic' defined in file [/tmp/jetty-0_0_0_0-8080-catalog-be_war-_-any-1713229416780379629/webapp/WEB-INF/classes/org/openecomp/sdc/be/components/impl/ArtifactsBusinessLogic.class]: Unsatisfied dependency expressed through constructor parameter 1; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'tosca-export-handler' defined in file [/tmp/jetty-0_0_0_0-8080-catalog-be_war-_-any-1713229416780379629/webapp/WEB-INF/classes/org/openecomp/sdc/be/tosca/ToscaExportHandler.class]: Unsatisfied dependency expressed through constructor parameter 3; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'policyExportParserImpl' defined in file [/tmp/jetty-0_0_0_0-8080-catalog-be_war-_-any-1713229416780379629/webapp/WEB-INF/classes/org/openecomp/sdc/be/tosca/PolicyExportParserImpl.class]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.openecomp.sdc.be.tosca.PolicyExportParserImpl]: Constructor threw exception; nested exception is org.openecomp.sdc.be.model.jsonjanusgraph.operations.exception.OperationException at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:800) at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:229) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:944) at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583) at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:401) at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:292) at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:103) at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1073) at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572) at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:1002) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746) at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379) at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449) at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414) at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916) at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288) at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.deploy.bindings.StandardStarter.processBinding(StandardStarter.java:46) at org.eclipse.jetty.deploy.AppLifeCycle.runBindings(AppLifeCycle.java:188) at org.eclipse.jetty.deploy.DeploymentManager.requestAppGoal(DeploymentManager.java:517) at org.eclipse.jetty.deploy.DeploymentManager.addApp(DeploymentManager.java:157) at org.eclipse.jetty.deploy.providers.ScanningAppProvider.fileAdded(ScanningAppProvider.java:173) at org.eclipse.jetty.deploy.providers.WebAppProvider.fileAdded(WebAppProvider.java:448) at org.eclipse.jetty.deploy.providers.ScanningAppProvider$1.fileAdded(ScanningAppProvider.java:66) at org.eclipse.jetty.util.Scanner.reportAddition(Scanner.java:785) at org.eclipse.jetty.util.Scanner.reportDifferences(Scanner.java:754) at org.eclipse.jetty.util.Scanner.scan(Scanner.java:641) at org.eclipse.jetty.util.Scanner.doStart(Scanner.java:540) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.deploy.providers.ScanningAppProvider.doStart(ScanningAppProvider.java:146) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.deploy.DeploymentManager.startAppProvider(DeploymentManager.java:605) at org.eclipse.jetty.deploy.DeploymentManager.doStart(DeploymentManager.java:252) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.server.Server.start(Server.java:423) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.server.Server.doStart(Server.java:387) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.xml.XmlConfiguration.lambda$main$3(XmlConfiguration.java:1907) at java.base/java.security.AccessController.doPrivileged(Native Method) at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1857) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.eclipse.jetty.start.Main.invokeMain(Main.java:218) at org.eclipse.jetty.start.Main.start(Main.java:491) at org.eclipse.jetty.start.Main.main(Main.java:77) Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'artifactBusinessLogic' defined in file [/tmp/jetty-0_0_0_0-8080-catalog-be_war-_-any-1713229416780379629/webapp/WEB-INF/classes/org/openecomp/sdc/be/components/impl/ArtifactsBusinessLogic.class]: Unsatisfied dependency expressed through constructor parameter 1; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'tosca-export-handler' defined in file [/tmp/jetty-0_0_0_0-8080-catalog-be_war-_-any-1713229416780379629/webapp/WEB-INF/classes/org/openecomp/sdc/be/tosca/ToscaExportHandler.class]: Unsatisfied dependency expressed through constructor parameter 3; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'policyExportParserImpl' defined in file [/tmp/jetty-0_0_0_0-8080-catalog-be_war-_-any-1713229416780379629/webapp/WEB-INF/classes/org/openecomp/sdc/be/tosca/PolicyExportParserImpl.class]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.openecomp.sdc.be.tosca.PolicyExportParserImpl]: Constructor threw exception; nested exception is org.openecomp.sdc.be.model.jsonjanusgraph.operations.exception.OperationException at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:800) at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:229) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1380) at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1300) at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887) at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791) ... 59 common frames omitted Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'tosca-export-handler' defined in file [/tmp/jetty-0_0_0_0-8080-catalog-be_war-_-any-1713229416780379629/webapp/WEB-INF/classes/org/openecomp/sdc/be/tosca/ToscaExportHandler.class]: Unsatisfied dependency expressed through constructor parameter 3; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'policyExportParserImpl' defined in file [/tmp/jetty-0_0_0_0-8080-catalog-be_war-_-any-1713229416780379629/webapp/WEB-INF/classes/org/openecomp/sdc/be/tosca/PolicyExportParserImpl.class]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.openecomp.sdc.be.tosca.PolicyExportParserImpl]: Constructor threw exception; nested exception is org.openecomp.sdc.be.model.jsonjanusgraph.operations.exception.OperationException at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:800) at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:229) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1380) at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1300) at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887) at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791) ... 73 common frames omitted Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'policyExportParserImpl' defined in file [/tmp/jetty-0_0_0_0-8080-catalog-be_war-_-any-1713229416780379629/webapp/WEB-INF/classes/org/openecomp/sdc/be/tosca/PolicyExportParserImpl.class]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.openecomp.sdc.be.tosca.PolicyExportParserImpl]: Constructor threw exception; nested exception is org.openecomp.sdc.be.model.jsonjanusgraph.operations.exception.OperationException at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:315) at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:296) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1380) at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1300) at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887) at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791) ... 87 common frames omitted Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.openecomp.sdc.be.tosca.PolicyExportParserImpl]: Constructor threw exception; nested exception is org.openecomp.sdc.be.model.jsonjanusgraph.operations.exception.OperationException at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:224) at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:117) at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:311) ... 101 common frames omitted Caused by: org.openecomp.sdc.be.model.jsonjanusgraph.operations.exception.OperationException: null at org.openecomp.sdc.be.model.jsonjanusgraph.operations.exception.ModelOperationExceptionSupplier.lambda$failedToRetrieveModels$4(ModelOperationExceptionSupplier.java:53) at org.openecomp.sdc.be.model.operations.impl.ModelOperation.findModelVerticesByCriteria(ModelOperation.java:239) at org.openecomp.sdc.be.model.operations.impl.ModelOperation.findModelsByCriteria(ModelOperation.java:224) at org.openecomp.sdc.be.model.operations.impl.ModelOperation.findAllModels(ModelOperation.java:213) at org.openecomp.sdc.be.model.operations.impl.DataTypeOperation.getAllDataTypeUidsToModels(DataTypeOperation.java:94) at org.openecomp.sdc.be.model.operations.impl.PropertyOperation.getAllDataTypes(PropertyOperation.java:1522) at org.openecomp.sdc.be.model.cache.ApplicationDataTypeCache.getAllDataTypesFromGraph(ApplicationDataTypeCache.java:165) at org.openecomp.sdc.be.model.cache.ApplicationDataTypeCache.getAll(ApplicationDataTypeCache.java:172) at org.openecomp.sdc.be.tosca.PolicyExportParserImpl.getDataTypes(PolicyExportParserImpl.java:67) at org.openecomp.sdc.be.tosca.PolicyExportParserImpl.(PolicyExportParserImpl.java:63) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) at java.base/java.lang.reflect.Constructor.newInstance(Unknown Source) at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:211) ... 103 common frames omitted 2022-04-26T06:03:45.779Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient ** open graph with /app/jetty/config/catalog-be/janusgraph.properties started 2022-04-26T06:03:45.780Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient openGraph : try to load file /app/jetty/config/catalog-be/janusgraph.properties 2022-04-26T06:03:45.784Z [JanusGraph-Reconnect-Task] INFO c.d.driver.core.SystemProperties com.datastax.driver.USE_NATIVE_CLOCK is defined, using value false 2022-04-26T06:03:45.785Z [JanusGraph-Reconnect-Task] INFO c.datastax.driver.core.ClockFactory Using java.lang.System clock to generate timestamps. 2022-04-26T06:03:45.797Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient createGraph : failed to open JanusGraph graph with configuration file: /app/jetty/config/catalog-be/janusgraph.properties 2022-04-26T06:03:48.779Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient ** open graph with /app/jetty/config/catalog-be/janusgraph.properties started 2022-04-26T06:03:48.779Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient openGraph : try to load file /app/jetty/config/catalog-be/janusgraph.properties 2022-04-26T06:03:48.782Z [JanusGraph-Reconnect-Task] INFO c.d.driver.core.SystemProperties com.datastax.driver.USE_NATIVE_CLOCK is defined, using value false 2022-04-26T06:03:48.783Z [JanusGraph-Reconnect-Task] INFO c.datastax.driver.core.ClockFactory Using java.lang.System clock to generate timestamps. 2022-04-26T06:03:48.792Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient createGraph : failed to open JanusGraph graph with configuration file: /app/jetty/config/catalog-be/janusgraph.properties 2022-04-26T06:03:51.779Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient ** open graph with /app/jetty/config/catalog-be/janusgraph.properties started 2022-04-26T06:03:51.780Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient openGraph : try to load file /app/jetty/config/catalog-be/janusgraph.properties 2022-04-26T06:03:51.782Z [JanusGraph-Reconnect-Task] INFO c.d.driver.core.SystemProperties com.datastax.driver.USE_NATIVE_CLOCK is defined, using value false 2022-04-26T06:03:51.782Z [JanusGraph-Reconnect-Task] INFO c.datastax.driver.core.ClockFactory Using java.lang.System clock to generate timestamps. 2022-04-26T06:03:51.793Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient createGraph : failed to open JanusGraph graph with configuration file: /app/jetty/config/catalog-be/janusgraph.properties 2022-04-26T06:03:54.779Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient ** open graph with /app/jetty/config/catalog-be/janusgraph.properties started 2022-04-26T06:03:54.780Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient openGraph : try to load file /app/jetty/config/catalog-be/janusgraph.properties 2022-04-26T06:03:54.783Z [JanusGraph-Reconnect-Task] INFO c.d.driver.core.SystemProperties com.datastax.driver.USE_NATIVE_CLOCK is defined, using value false 2022-04-26T06:03:54.783Z [JanusGraph-Reconnect-Task] INFO c.datastax.driver.core.ClockFactory Using java.lang.System clock to generate timestamps. 2022-04-26T06:03:54.794Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient createGraph : failed to open JanusGraph graph with configuration file: /app/jetty/config/catalog-be/janusgraph.properties 2022-04-26T06:03:57.779Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient ** open graph with /app/jetty/config/catalog-be/janusgraph.properties started 2022-04-26T06:03:57.780Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient openGraph : try to load file /app/jetty/config/catalog-be/janusgraph.properties 2022-04-26T06:03:57.783Z [JanusGraph-Reconnect-Task] INFO c.d.driver.core.SystemProperties com.datastax.driver.USE_NATIVE_CLOCK is defined, using value false 2022-04-26T06:03:57.783Z [JanusGraph-Reconnect-Task] INFO c.datastax.driver.core.ClockFactory Using java.lang.System clock to generate timestamps. 2022-04-26T06:03:57.793Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient createGraph : failed to open JanusGraph graph with configuration file: /app/jetty/config/catalog-be/janusgraph.properties 2022-04-26T06:04:00.779Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient ** open graph with /app/jetty/config/catalog-be/janusgraph.properties started 2022-04-26T06:04:00.779Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient openGraph : try to load file /app/jetty/config/catalog-be/janusgraph.properties 2022-04-26T06:04:00.782Z [JanusGraph-Reconnect-Task] INFO c.d.driver.core.SystemProperties com.datastax.driver.USE_NATIVE_CLOCK is defined, using value false 2022-04-26T06:04:00.782Z [JanusGraph-Reconnect-Task] INFO c.datastax.driver.core.ClockFactory Using java.lang.System clock to generate timestamps. 2022-04-26T06:04:00.791Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient createGraph : failed to open JanusGraph graph with configuration file: /app/jetty/config/catalog-be/janusgraph.properties 2022-04-26T06:04:03.779Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient ** open graph with /app/jetty/config/catalog-be/janusgraph.properties started 2022-04-26T06:04:03.780Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient openGraph : try to load file /app/jetty/config/catalog-be/janusgraph.properties 2022-04-26T06:04:03.783Z [JanusGraph-Reconnect-Task] INFO c.d.driver.core.SystemProperties com.datastax.driver.USE_NATIVE_CLOCK is defined, using value false 2022-04-26T06:04:03.783Z [JanusGraph-Reconnect-Task] INFO c.datastax.driver.core.ClockFactory Using java.lang.System clock to generate timestamps. 2022-04-26T06:04:03.798Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient createGraph : failed to open JanusGraph graph with configuration file: /app/jetty/config/catalog-be/janusgraph.properties 2022-04-26T06:04:06.779Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient ** open graph with /app/jetty/config/catalog-be/janusgraph.properties started 2022-04-26T06:04:06.780Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient openGraph : try to load file /app/jetty/config/catalog-be/janusgraph.properties 2022-04-26T06:04:06.791Z [JanusGraph-Reconnect-Task] INFO c.d.driver.core.SystemProperties com.datastax.driver.USE_NATIVE_CLOCK is defined, using value false 2022-04-26T06:04:06.792Z [JanusGraph-Reconnect-Task] INFO c.datastax.driver.core.ClockFactory Using java.lang.System clock to generate timestamps. 2022-04-26T06:04:07.305Z [JanusGraph-Reconnect-Task] INFO c.d.d.c.p.DCAwareRoundRobinPolicy Using provided data-center name 'Pod' for DCAwareRoundRobinPolicy 2022-04-26T06:04:07.306Z [JanusGraph-Reconnect-Task] INFO com.datastax.driver.core.Cluster New Cassandra host /10.233.72.7:9042 added 2022-04-26T06:04:07.306Z [JanusGraph-Reconnect-Task] INFO com.datastax.driver.core.Cluster New Cassandra host cassandra.onap/10.233.65.18:9042 added 2022-04-26T06:04:07.306Z [JanusGraph-Reconnect-Task] INFO com.datastax.driver.core.Cluster New Cassandra host /10.233.68.64:9042 added 2022-04-26T06:04:07.316Z [JanusGraph Cluster-nio-worker-0] WARN c.d.driver.core.HostConnectionPool Error creating connection to /10.233.72.7:9042 com.datastax.driver.core.exceptions.TransportException: [/10.233.72.7:9042] Cannot connect at com.datastax.driver.core.Connection$1.operationComplete(Connection.java:193) at com.datastax.driver.core.Connection$1.operationComplete(Connection.java:163) at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578) at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:571) at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:550) at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491) at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616) at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609) at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:470) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: /10.233.72.7:9042 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at io.netty.channel.unix.Errors.newConnectException0(Errors.java:155) at io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128) at io.netty.channel.unix.Socket.finishConnect(Socket.java:320) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:470) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:04:07.317Z [JanusGraph Cluster-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to /10.233.72.7:9042 com.datastax.driver.core.exceptions.ConnectionException: [/10.233.72.7:9042] Pool was closed during initialization at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:173) at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:158) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1080) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.google.common.util.concurrent.CollectionFuture.handleAllCompleted(CollectionFuture.java:69) at com.google.common.util.concurrent.AggregateFuture.processCompleted(AggregateFuture.java:282) at com.google.common.util.concurrent.AggregateFuture.decrementCountAndMaybeComplete(AggregateFuture.java:264) at com.google.common.util.concurrent.AggregateFuture.access$200(AggregateFuture.java:42) at com.google.common.util.concurrent.AggregateFuture$1.run(AggregateFuture.java:147) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:100) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:53) at com.datastax.driver.core.Connection$1.operationComplete(Connection.java:191) at com.datastax.driver.core.Connection$1.operationComplete(Connection.java:163) at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578) at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:571) at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:550) at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491) at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616) at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609) at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:470) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:04:07.697Z [JanusGraph-Reconnect-Task] WARN o.j.d.c.b.ReadConfigurationBuilder Local setting storage.cql.replication-strategy-options=[Pod, 3] (Type: FIXED) is overridden by globally managed value ([Pod, 3]). Use the ManagementSystem interface instead of the local configuration to control this setting. 2022-04-26T06:04:09.729Z [JanusGraph-Reconnect-Task] INFO o.j.g.i.UniqueInstanceIdRetriever Generated unique-instance-id=0ae94129116-onap-sdc-be-6b55bdfcb4-c9qbb1 2022-04-26T06:04:09.744Z [JanusGraph-Reconnect-Task] INFO c.d.driver.core.SystemProperties com.datastax.driver.USE_NATIVE_CLOCK is defined, using value false 2022-04-26T06:04:09.745Z [JanusGraph-Reconnect-Task] INFO c.datastax.driver.core.ClockFactory Using java.lang.System clock to generate timestamps. 2022-04-26T06:04:09.985Z [JanusGraph-Reconnect-Task] INFO c.d.d.c.p.DCAwareRoundRobinPolicy Using provided data-center name 'Pod' for DCAwareRoundRobinPolicy 2022-04-26T06:04:09.985Z [JanusGraph-Reconnect-Task] INFO com.datastax.driver.core.Cluster New Cassandra host /10.233.72.7:9042 added 2022-04-26T06:04:09.985Z [JanusGraph-Reconnect-Task] INFO com.datastax.driver.core.Cluster New Cassandra host cassandra.onap/10.233.65.18:9042 added 2022-04-26T06:04:09.985Z [JanusGraph-Reconnect-Task] INFO com.datastax.driver.core.Cluster New Cassandra host /10.233.68.64:9042 added 2022-04-26T06:04:09.995Z [JanusGraph Cluster-nio-worker-0] WARN c.d.driver.core.HostConnectionPool Error creating connection to /10.233.72.7:9042 com.datastax.driver.core.exceptions.TransportException: [/10.233.72.7:9042] Cannot connect at com.datastax.driver.core.Connection$1.operationComplete(Connection.java:193) at com.datastax.driver.core.Connection$1.operationComplete(Connection.java:163) at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578) at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:571) at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:550) at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491) at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616) at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609) at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:470) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: /10.233.72.7:9042 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at io.netty.channel.unix.Errors.newConnectException0(Errors.java:155) at io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128) at io.netty.channel.unix.Socket.finishConnect(Socket.java:320) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:470) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:04:09.996Z [JanusGraph Cluster-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to /10.233.72.7:9042 com.datastax.driver.core.exceptions.ConnectionException: [/10.233.72.7:9042] Pool was closed during initialization at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:173) at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:158) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1080) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.google.common.util.concurrent.CollectionFuture.handleAllCompleted(CollectionFuture.java:69) at com.google.common.util.concurrent.AggregateFuture.processCompleted(AggregateFuture.java:282) at com.google.common.util.concurrent.AggregateFuture.decrementCountAndMaybeComplete(AggregateFuture.java:264) at com.google.common.util.concurrent.AggregateFuture.access$200(AggregateFuture.java:42) at com.google.common.util.concurrent.AggregateFuture$1.run(AggregateFuture.java:147) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:100) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:53) at com.datastax.driver.core.Connection$1.operationComplete(Connection.java:191) at com.datastax.driver.core.Connection$1.operationComplete(Connection.java:163) at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578) at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:571) at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:550) at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491) at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616) at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609) at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:470) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:04:10.145Z [JanusGraph-Reconnect-Task] INFO org.janusgraph.diskstorage.Backend Initiated backend operations thread pool of size 2 2022-04-26T06:04:10.282Z [JanusGraph-Reconnect-Task] INFO org.janusgraph.diskstorage.Backend Configuring total store cache size: 544668788 2022-04-26T06:04:10.495Z [JanusGraph-Reconnect-Task] INFO o.j.diskstorage.log.kcvs.KCVSLog Loaded unidentified ReadMarker start time 2022-04-26T06:04:10.495274Z into org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller@51447501 2022-04-26T06:04:10.663Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient ** JanusGraph graph created 2022-04-26T06:04:10.664Z [JanusGraph-Reconnect-Task] INFO o.o.s.b.d.j.JanusGraphClient ** Cancelling JanusGraph reconnect task 2022-04-26T06:14:11.269Z [main] WARN o.o.s.c.util.YamlToObjectConverter RequestID=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= The file /app/jetty/config/catalog-be/neo4j-errors-configuration.yaml cannot be found. Ignore reading configuration. 2022-04-26T06:14:11.392Z [main] INFO o.o.s.be.listen.BEAppContextListener RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ASDC version from MANIFEST is 1.10.4 2022-04-26T06:14:11.454Z [main] INFO o.o.s.b.m.BeMonitoringService RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= BE monitoring service is disabled 2022-04-26T06:14:11.456Z [main] INFO o.s.web.context.ContextLoader RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Root WebApplicationContext: initialization started 2022-04-26T06:14:14.063Z [main] INFO o.o.s.b.d.j.JanusGraphClient RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ** JanusGraphClient created 2022-04-26T06:14:14.068Z [main] INFO o.o.s.b.d.j.JanusGraphClient RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ** createGraph started ** 2022-04-26T06:14:14.068Z [main] INFO o.o.s.b.d.j.JanusGraphClient RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ** open graph with /app/jetty/config/catalog-be/janusgraph.properties started 2022-04-26T06:14:14.068Z [main] INFO o.o.s.b.d.j.JanusGraphClient RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= openGraph : try to load file /app/jetty/config/catalog-be/janusgraph.properties 2022-04-26T06:14:14.574Z [main] INFO com.datastax.driver.core RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= DataStax Java driver 3.8.0 for Apache Cassandra 2022-04-26T06:14:14.579Z [main] INFO c.d.driver.core.GuavaCompatibility RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Detected Guava >= 19 in the classpath, using modern compatibility layer 2022-04-26T06:14:14.618Z [main] INFO c.d.driver.core.SystemProperties RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= com.datastax.driver.USE_NATIVE_CLOCK is defined, using value false 2022-04-26T06:14:14.619Z [main] INFO c.datastax.driver.core.ClockFactory RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Using java.lang.System clock to generate timestamps. 2022-04-26T06:14:14.825Z [main] INFO com.datastax.driver.core.NettyUtil RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Found Netty's native epoll transport in the classpath, using it 2022-04-26T06:14:26.520Z [main] INFO c.d.d.c.p.DCAwareRoundRobinPolicy RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Using provided data-center name 'Pod' for DCAwareRoundRobinPolicy 2022-04-26T06:14:26.522Z [main] INFO com.datastax.driver.core.Cluster RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= New Cassandra host cassandra.onap/10.233.72.7:9042 added 2022-04-26T06:14:26.522Z [main] INFO com.datastax.driver.core.Cluster RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= New Cassandra host /10.233.65.18:9042 added 2022-04-26T06:14:26.523Z [main] INFO com.datastax.driver.core.Cluster RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= New Cassandra host /10.233.68.64:9042 added 2022-04-26T06:14:40.589Z [main] WARN o.j.d.c.b.ReadConfigurationBuilder RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Local setting storage.cql.replication-strategy-options=[Pod, 3] (Type: FIXED) is overridden by globally managed value ([Pod, 3]). Use the ManagementSystem interface instead of the local configuration to control this setting. 2022-04-26T06:14:42.800Z [main] INFO o.j.g.i.UniqueInstanceIdRetriever RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Generated unique-instance-id=0ae94129116-onap-sdc-be-6b55bdfcb4-c9qbb1 2022-04-26T06:14:42.816Z [main] INFO c.d.driver.core.SystemProperties RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= com.datastax.driver.USE_NATIVE_CLOCK is defined, using value false 2022-04-26T06:14:42.817Z [main] INFO c.datastax.driver.core.ClockFactory RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Using java.lang.System clock to generate timestamps. 2022-04-26T06:14:53.723Z [main] INFO c.d.d.c.p.DCAwareRoundRobinPolicy RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Using provided data-center name 'Pod' for DCAwareRoundRobinPolicy 2022-04-26T06:14:53.724Z [main] INFO com.datastax.driver.core.Cluster RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= New Cassandra host cassandra.onap/10.233.72.7:9042 added 2022-04-26T06:14:53.724Z [main] INFO com.datastax.driver.core.Cluster RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= New Cassandra host /10.233.65.18:9042 added 2022-04-26T06:14:53.724Z [main] INFO com.datastax.driver.core.Cluster RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= New Cassandra host /10.233.68.64:9042 added 2022-04-26T06:15:04.548Z [main] INFO org.janusgraph.diskstorage.Backend RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Initiated backend operations thread pool of size 2 2022-04-26T06:15:12.647Z [main] INFO org.janusgraph.diskstorage.Backend RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Configuring total store cache size: 674590508 2022-04-26T06:15:21.163Z [main] INFO o.j.diskstorage.log.kcvs.KCVSLog RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Loaded unidentified ReadMarker start time 2022-04-26T06:15:21.163426Z into org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller@5c59dfc7 2022-04-26T06:15:25.041Z [main] INFO o.o.s.b.d.j.JanusGraphClient RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ** JanusGraph graph created 2022-04-26T06:15:29.774Z [JanusGraphID(20)(3)[0]] WARN o.j.d.i.ConsistentKeyIDAuthority Temporary storage exception while acquiring id block - retrying in PT0.6S: {} org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.mutate(CQLKeyColumnValueStore.java:364) at org.janusgraph.diskstorage.idmanagement.ConsistentKeyIDAuthority.lambda$getIDBlock$1(ConsistentKeyIDAuthority.java:261) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.idmanagement.ConsistentKeyIDAuthority.getIDBlock(ConsistentKeyIDAuthority.java:260) at org.janusgraph.graphdb.database.idassigner.StandardIDPool$IDBlockGetter.call(StandardIDPool.java:288) at org.janusgraph.graphdb.database.idassigner.StandardIDPool$IDBlockGetter.call(StandardIDPool.java:255) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency QUORUM (2 replica were required but only 1 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency QUORUM (2 replica were required but only 1 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency QUORUM (2 replica were required but only 1 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:15:33.547Z [main] INFO o.o.s.b.d.j.JanusGraphGenericDao RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ** JanusGraphGenericDao created 2022-04-26T06:15:33.661Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= All sdc keyspaces are : [sdcartifact, sdcrepository, sdcaudit] 2022-04-26T06:15:33.663Z [main] INFO o.o.s.b.d.c.schema.SdcSchemaUtils RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Connecting to node: [cassandra.onap] port: 9042. 2022-04-26T06:15:33.677Z [main] INFO o.o.s.b.d.c.schema.SdcSchemaUtils RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Connection timeout in seconds : 120 2022-04-26T06:15:33.677Z [main] INFO o.o.s.b.d.c.schema.SdcSchemaUtils RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= SocketConnectTimeout was provided, setting Cassandra client to use SocketConnectTimeout: 20000 . 2022-04-26T06:15:33.678Z [main] INFO o.o.s.b.d.c.schema.SdcSchemaUtils RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= SocketReadTimeout was provided, setting Cassandra client to use SocketReadTimeout: 20000 . 2022-04-26T06:15:33.678Z [main] INFO o.o.s.b.d.c.schema.SdcSchemaUtils RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= localDatacenter was provided, setting Cassndra clint to use datacenter: Pod as local. 2022-04-26T06:15:33.679Z [main] INFO c.d.driver.core.SystemProperties RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= com.datastax.driver.USE_NATIVE_CLOCK is defined, using value false 2022-04-26T06:15:33.679Z [main] INFO c.datastax.driver.core.ClockFactory RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Using java.lang.System clock to generate timestamps. 2022-04-26T06:15:33.679Z [main] INFO o.o.s.b.d.c.schema.SdcSchemaUtils RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ** cluster created 2022-04-26T06:15:33.680Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= creating cluster for Cassandra Health Check. 2022-04-26T06:15:33.762Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.233.65.41 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2022-04-26T06:15:34.709Z [main] INFO c.d.d.c.p.DCAwareRoundRobinPolicy RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Using provided data-center name 'Pod' for DCAwareRoundRobinPolicy 2022-04-26T06:15:34.710Z [main] INFO com.datastax.driver.core.Cluster RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= New Cassandra host cassandra.onap/10.233.72.7:9042 added 2022-04-26T06:15:34.710Z [main] INFO com.datastax.driver.core.Cluster RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= New Cassandra host cassandra.onap/10.233.65.18:9042 added 2022-04-26T06:15:34.711Z [main] INFO com.datastax.driver.core.Cluster RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= New Cassandra host cassandra.onap/10.233.68.64:9042 added 2022-04-26T06:15:34.711Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Cluster Metadata: com.datastax.driver.core.Metadata@d03cdc7 2022-04-26T06:15:34.712Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= keyspace : sdcartifact , replication: {Pod=3, class=org.apache.cassandra.locator.NetworkTopologyStrategy} 2022-04-26T06:15:34.712Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= keyspace : sdcrepository , replication: {Pod=3, class=org.apache.cassandra.locator.NetworkTopologyStrategy} 2022-04-26T06:15:34.713Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= keyspace : sdcaudit , replication: {Pod=3, class=org.apache.cassandra.locator.NetworkTopologyStrategy} 2022-04-26T06:15:34.714Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= maxReplication Factor is: 3 2022-04-26T06:15:34.714Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= localQuorum is: 2 2022-04-26T06:15:34.714Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= Health Check formula : Replication Factor – Local_Quorum = 1 2022-04-26T06:15:34.730Z [main] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ECOMP Portal health check configuration is missing. 2022-04-26T06:15:34.739Z [main] INFO o.o.s.b.s.d.SwitchoverDetector RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Enter init method of SwitchoverDetector 2022-04-26T06:15:34.740Z [main] INFO o.o.s.b.s.d.SwitchoverDetector RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null switchover detector service is disabled 2022-04-26T06:15:37.534Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.233.65.41 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2022-04-26T06:15:37.536Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2022-04-26T06:15:42.746Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.233.65.41 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2022-04-26T06:15:46.532Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.233.65.41 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2022-04-26T06:15:46.533Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2022-04-26T06:15:47.939Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null creating cluster for Cassandra for monitoring. 2022-04-26T06:15:47.940Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null The cassandra session is com.datastax.driver.core.SessionManager@978475b 2022-04-26T06:15:47.940Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null The number of cassandra nodes is:3 2022-04-26T06:15:47.943Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null The cassandra down nodes number is 0 2022-04-26T06:15:48.542Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.233.65.41 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2022-04-26T06:15:51.647Z [main] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:15:51.649Z [main] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.init(HealthCheckBusinessLogic.java:106) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:389) at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:333) at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:157) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:440) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1796) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:620) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:944) at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583) at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:401) at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:292) at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:103) at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1073) at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572) at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:1002) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746) at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379) at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449) at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414) at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916) at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288) at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.deploy.bindings.StandardStarter.processBinding(StandardStarter.java:46) at org.eclipse.jetty.deploy.AppLifeCycle.runBindings(AppLifeCycle.java:188) at org.eclipse.jetty.deploy.DeploymentManager.requestAppGoal(DeploymentManager.java:517) at org.eclipse.jetty.deploy.DeploymentManager.addApp(DeploymentManager.java:157) at org.eclipse.jetty.deploy.providers.ScanningAppProvider.fileAdded(ScanningAppProvider.java:173) at org.eclipse.jetty.deploy.providers.WebAppProvider.fileAdded(WebAppProvider.java:448) at org.eclipse.jetty.deploy.providers.ScanningAppProvider$1.fileAdded(ScanningAppProvider.java:66) at org.eclipse.jetty.util.Scanner.reportAddition(Scanner.java:785) at org.eclipse.jetty.util.Scanner.reportDifferences(Scanner.java:754) at org.eclipse.jetty.util.Scanner.scan(Scanner.java:641) at org.eclipse.jetty.util.Scanner.doStart(Scanner.java:540) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.deploy.providers.ScanningAppProvider.doStart(ScanningAppProvider.java:146) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.deploy.DeploymentManager.startAppProvider(DeploymentManager.java:605) at org.eclipse.jetty.deploy.DeploymentManager.doStart(DeploymentManager.java:252) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.server.Server.start(Server.java:423) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.server.Server.doStart(Server.java:387) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.xml.XmlConfiguration.lambda$main$3(XmlConfiguration.java:1907) at java.base/java.security.AccessController.doPrivileged(Native Method) at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1857) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.eclipse.jetty.start.Main.invokeMain(Main.java:218) at org.eclipse.jetty.start.Main.start(Main.java:491) at org.eclipse.jetty.start.Main.main(Main.java:77) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 72 common frames omitted 2022-04-26T06:15:51.758Z [main] INFO o.o.s.b.dao.janusgraph.JanusGraphDao RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ** JanusGraphDao created 2022-04-26T06:15:51.849Z [main] INFO o.o.s.b.d.cassandra.CassandraClient RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null SocketConnectTimeout was provided, setting Cassandra client to use SocketConnectTimeout: 20000 . 2022-04-26T06:15:51.850Z [main] INFO o.o.s.b.d.cassandra.CassandraClient RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null SocketReadTimeout was provided, setting Cassandra client to use SocketReadTimeout: 20000 . 2022-04-26T06:15:51.851Z [main] INFO o.o.s.b.d.cassandra.CassandraClient RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null localDatacenter was provided, setting Cassndra clint to use datacenter: Pod as local. 2022-04-26T06:15:51.851Z [main] INFO c.d.driver.core.SystemProperties RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null com.datastax.driver.USE_NATIVE_CLOCK is defined, using value false 2022-04-26T06:15:51.851Z [main] INFO c.datastax.driver.core.ClockFactory RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Using java.lang.System clock to generate timestamps. 2022-04-26T06:15:51.851Z [main] INFO o.o.s.b.d.cassandra.CassandraClient RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ** CassandraClient created 2022-04-26T06:16:01.201Z [main] INFO c.d.d.c.p.DCAwareRoundRobinPolicy RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Using provided data-center name 'Pod' for DCAwareRoundRobinPolicy 2022-04-26T06:16:01.202Z [main] INFO com.datastax.driver.core.Cluster RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null New Cassandra host cassandra.onap/10.233.72.7:9042 added 2022-04-26T06:16:01.202Z [main] INFO com.datastax.driver.core.Cluster RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null New Cassandra host cassandra.onap/10.233.65.18:9042 added 2022-04-26T06:16:01.203Z [main] INFO com.datastax.driver.core.Cluster RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null New Cassandra host cassandra.onap/10.233.68.64:9042 added 2022-04-26T06:16:06.145Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=475152fd-1fca-4e22-9782-a05d1facf403 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:16:06.146Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=475152fd-1fca-4e22-9782-a05d1facf403 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6be9051e 2022-04-26T06:16:06.146Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=475152fd-1fca-4e22-9782-a05d1facf403 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:16:06.146Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=475152fd-1fca-4e22-9782-a05d1facf403 PartnerName=SDC.BE The cassandra down nodes number is 0 2022-04-26T06:16:08.073Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=475152fd-1fca-4e22-9782-a05d1facf403 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:16:08.075Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=475152fd-1fca-4e22-9782-a05d1facf403 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:16:09.449Z [main] INFO o.o.s.b.d.c.ToscaModelImportCassandraDao RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null org.openecomp.sdc.be.dao.cassandra.ToscaModelImportCassandraDao successfully initialized 2022-04-26T06:16:13.442Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3e66e536-e96f-43f4-89f2-09ca7b806495 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:16:13.443Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3e66e536-e96f-43f4-89f2-09ca7b806495 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@402a2d75 2022-04-26T06:16:13.443Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3e66e536-e96f-43f4-89f2-09ca7b806495 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:16:13.444Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3e66e536-e96f-43f4-89f2-09ca7b806495 PartnerName=SDC.BE The cassandra down nodes number is 0 2022-04-26T06:16:16.452Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3e66e536-e96f-43f4-89f2-09ca7b806495 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:16:16.453Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3e66e536-e96f-43f4-89f2-09ca7b806495 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:16:16.455Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=3e66e536-e96f-43f4-89f2-09ca7b806495 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:16:16.455Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=3e66e536-e96f-43f4-89f2-09ca7b806495 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:16:18.583Z [main] INFO o.o.s.b.d.c.AuditCassandraDao RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ** AuditCassandraDao created 2022-04-26T06:16:18.633Z [main] INFO o.o.s.b.c.d.e.DmaapClientFactory RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:16:18.723Z [main] ERROR o.o.s.be.catalog.impl.DmaapProducer RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:16:30.531Z [main] INFO o.o.s.b.d.c.ArtifactCassandraDao RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ** ArtifactCassandraDao created 2022-04-26T06:16:31.740Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e48ffd5-6db9-4c8c-a8b8-edba7e4aed02 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:16:31.741Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e48ffd5-6db9-4c8c-a8b8-edba7e4aed02 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6f42f16 2022-04-26T06:16:31.741Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e48ffd5-6db9-4c8c-a8b8-edba7e4aed02 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:16:31.741Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e48ffd5-6db9-4c8c-a8b8-edba7e4aed02 PartnerName=SDC.BE The cassandra down nodes number is 0 2022-04-26T06:16:34.058Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=0e48ffd5-6db9-4c8c-a8b8-edba7e4aed02 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:16:34.059Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=0e48ffd5-6db9-4c8c-a8b8-edba7e4aed02 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:16:35.050Z [main] INFO o.o.s.b.d.c.SdcSchemaFilesCassandraDao RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ** SdcSchemaFilesCassandraDao created 2022-04-26T06:16:35.352Z [main] WARN o.o.s.b.c.s.CertificateManagerImpl RequestID=df6feeb4-f8dd-45f7-a1ab-de774757b273 ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Environment variable 'SDC_CERT_DIR' was not provided. Could not load certificates. 2022-04-26T06:16:41.047Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f054f187-910d-4bd1-8d9f-211834662ee1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:16:41.048Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f054f187-910d-4bd1-8d9f-211834662ee1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4505f851 2022-04-26T06:16:41.048Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f054f187-910d-4bd1-8d9f-211834662ee1 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:16:41.048Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f054f187-910d-4bd1-8d9f-211834662ee1 PartnerName=SDC.BE The cassandra down nodes number is 0 2022-04-26T06:16:49.945Z [main] INFO c.att.nsa.apiClient.http.HttpClient RequestID=8e073b9f-a929-431a-834b-d48c04fb9c1c PartnerName=SDC.BE POST http://message-router.onap:3904/apiKeys/create (anonymous) ... 2022-04-26T06:16:50.041Z [main] INFO c.att.nsa.apiClient.http.HttpClient RequestID=8e073b9f-a929-431a-834b-d48c04fb9c1c PartnerName=SDC.BE \t--> HTTP/1.1 200 OK 2022-04-26T06:16:50.240Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d10efe9d-b3d8-42f0-997a-faebbd17a3a1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:16:50.241Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d10efe9d-b3d8-42f0-997a-faebbd17a3a1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4a51aa08 2022-04-26T06:16:50.241Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d10efe9d-b3d8-42f0-997a-faebbd17a3a1 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:16:50.241Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d10efe9d-b3d8-42f0-997a-faebbd17a3a1 PartnerName=SDC.BE The cassandra down nodes number is 0 2022-04-26T06:16:50.355Z [main] INFO o.o.s.b.c.d.e.EnvironmentsEngine RequestID=8e073b9f-a929-431a-834b-d48c04fb9c1c PartnerName=SDC.BE Environments engine is disabled 2022-04-26T06:16:50.469Z [pool-17-thread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/topics (anonymous) ... 2022-04-26T06:16:50.492Z [pool-17-thread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:16:50.503Z [pool-17-thread-1] WARN c.att.nsa.apiClient.http.HttpClient POST http://message-router.onap:3904/topics/create will send credentials over a clear channel. 2022-04-26T06:16:50.504Z [pool-17-thread-1] INFO c.att.nsa.apiClient.http.HttpClient POST http://message-router.onap:3904/topics/create (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:16:50.672Z [pool-17-thread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:16:51.145Z [pool-17-thread-1] INFO com.datastax.driver.core.utils.UUIDs PID obtained through native call to getpid(): 116 2022-04-26T06:16:51.939Z [main] INFO o.o.s.b.c.s.AsdcComponentsCleanerTask RequestID=8e073b9f-a929-431a-834b-d48c04fb9c1c PartnerName=SDC.BE ServerIPAddress=10.233.65.41 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Enter init method of AsdcComponentsCleaner 2022-04-26T06:16:51.940Z [main] INFO o.o.s.b.c.s.AsdcComponentsCleanerTask RequestID=8e073b9f-a929-431a-834b-d48c04fb9c1c PartnerName=SDC.BE ServerIPAddress=10.233.65.41 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb End init method of AsdcComponentsCleaner 2022-04-26T06:16:51.985Z [main] INFO o.s.b.f.a.AutowiredAnnotationBeanPostProcessor RequestID=8e073b9f-a929-431a-834b-d48c04fb9c1c PartnerName=SDC.BE ServerIPAddress=10.233.65.41 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Autowired annotation is not supported on static methods: public static void org.openecomp.sdc.be.components.impl.ImportUtils.setComponentsUtils(org.openecomp.sdc.be.impl.ComponentsUtils) 2022-04-26T06:16:52.363Z [pool-17-thread-1] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "CreateDistributionTopic" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-NOTIF-TOPIC-AUTO" STATUS = "CREATED" PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/producers/jsVxexVZMKFmNLbu will send credentials over a clear channel. 2022-04-26T06:16:52.364Z [pool-17-thread-1] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "CreateDistributionTopic" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-NOTIF-TOPIC-AUTO" STATUS = "CREATED" PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/producers/jsVxexVZMKFmNLbu (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:16:52.396Z [pool-17-thread-1] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "CreateDistributionTopic" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-NOTIF-TOPIC-AUTO" STATUS = "CREATED" \t--> HTTP/1.1 200 OK 2022-04-26T06:16:52.448Z [pool-17-thread-1] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-NOTIF-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "jsVxexVZMKFmNLbu" STATUS = "200" POST http://message-router.onap:3904/topics/create will send credentials over a clear channel. 2022-04-26T06:16:52.449Z [pool-17-thread-1] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-NOTIF-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "jsVxexVZMKFmNLbu" STATUS = "200" POST http://message-router.onap:3904/topics/create (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:16:52.566Z [pool-17-thread-1] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-NOTIF-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "jsVxexVZMKFmNLbu" STATUS = "200" \t--> HTTP/1.1 200 OK 2022-04-26T06:16:52.653Z [pool-17-thread-1] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "CreateDistributionTopic" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" STATUS = "CREATED" PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/consumers/jsVxexVZMKFmNLbu will send credentials over a clear channel. 2022-04-26T06:16:52.654Z [pool-17-thread-1] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "CreateDistributionTopic" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" STATUS = "CREATED" PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/consumers/jsVxexVZMKFmNLbu (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:16:52.674Z [pool-17-thread-1] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "CreateDistributionTopic" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" STATUS = "CREATED" \t--> HTTP/1.1 200 OK 2022-04-26T06:16:52.686Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:16:52.686Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:16:52.687Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:16:58.158Z [main] INFO o.o.s.b.d.cassandra.FeatureToggleDao RequestID=8e073b9f-a929-431a-834b-d48c04fb9c1c PartnerName=SDC.BE ServerIPAddress=10.233.65.41 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ** FeatureToggleDao created 2022-04-26T06:16:58.425Z [main] INFO o.s.web.context.ContextLoader RequestID=8e073b9f-a929-431a-834b-d48c04fb9c1c PartnerName=SDC.BE ServerIPAddress=10.233.65.41 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Root WebApplicationContext initialized in 166969 ms 2022-04-26T06:17:01.244Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8bf1781-c47d-46ab-94db-1d1228b4ecc1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:17:01.246Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8bf1781-c47d-46ab-94db-1d1228b4ecc1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@484518fd 2022-04-26T06:17:01.246Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8bf1781-c47d-46ab-94db-1d1228b4ecc1 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:17:01.247Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8bf1781-c47d-46ab-94db-1d1228b4ecc1 PartnerName=SDC.BE The cassandra down nodes number is 0 2022-04-26T06:17:01.982Z [main] INFO o.a.c.b.FluentPropertyBeanIntrospector RequestID=8e073b9f-a929-431a-834b-d48c04fb9c1c PartnerName=SDC.BE ServerIPAddress=10.233.65.41 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Error when creating PropertyDescriptor for public final void org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)! Ignoring this property. 2022-04-26T06:17:02.005Z [main] INFO o.a.c.b.FluentPropertyBeanIntrospector RequestID=8e073b9f-a929-431a-834b-d48c04fb9c1c PartnerName=SDC.BE ServerIPAddress=10.233.65.41 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Error when creating PropertyDescriptor for public final void org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)! Ignoring this property. 2022-04-26T06:17:02.048Z [main] INFO o.a.c.b.FluentPropertyBeanIntrospector RequestID=8e073b9f-a929-431a-834b-d48c04fb9c1c PartnerName=SDC.BE ServerIPAddress=10.233.65.41 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Error when creating PropertyDescriptor for public final void org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)! Ignoring this property. 2022-04-26T06:17:05.322Z [main] INFO o.h.validator.internal.util.Version RequestID=8e073b9f-a929-431a-834b-d48c04fb9c1c PartnerName=SDC.BE ServerIPAddress=10.233.65.41 ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb HV000001: Hibernate Validator 6.1.6.Final 2022-04-26T06:17:05.368Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:17:05.378Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:17:05.380Z [UEB-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL ErrorCategory=INFO RequestID=null ServiceName=uebHealthCheck TargetServiceName=null ErrorCode=200 TargetEntity=uebHealthCheck ASDC Backend connection recovery to UEB Cluster. 2022-04-26T06:17:06.528Z [qtp2085002312-13] INFO com.jcabi.aspects.aj.NamedThreads RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/healthCheck EntryTimestamp=2022-04-26T06:17:06.432Z InvocationID=f3500d6b-8334-4d95-9126-32a60b27d7ca PartnerName=kube-probe/1.22 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb jcabi-aspects 0.20.1/f0aff93 started new daemon thread jcabi-loggable for watching of @Loggable annotated methods 2022-04-26T06:17:08.353Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:17:13.265Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6671bc05-89e2-4c0e-9fc9-311846e68260 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=1 Severity=0 PartnerName=aai auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.67.16 StatusCode=INPROGRESS RemoteHost=10.233.67.16 ServerIPAddress=10.233.65.41 ServiceName=v1/artifactTypes ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:13.260391Z InvocationID=7cf6d3a4-2c72-4975-8a6e-cb7c25bae562 LogTimestamp=2022-04-26T06:17:13.261924Z Entering 2022-04-26T06:17:13.266Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6671bc05-89e2-4c0e-9fc9-311846e68260 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=1 Severity=0 PartnerName=aai auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.67.16 StatusCode=INPROGRESS userId=null RemoteHost=10.233.67.16 ServerIPAddress=10.233.65.41 ServiceName=v1/artifactTypes ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:13.260391Z InvocationID=7cf6d3a4-2c72-4975-8a6e-cb7c25bae562 LogTimestamp=2022-04-26T06:17:13.261924Z localAddr=10.233.65.41 remoteAddr=10.233.67.16 GET /sdc/v1/artifactTypes HTTP/1.1 2022-04-26T06:17:13.432Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=163 PartnerName=Apache-HttpClient/4.5.10 (Java/11.0.9) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/artifactTypes EntryTimestamp=2022-04-26T06:17:13.268Z InvocationID=744276d9-0832-4b3a-8e16-cfcf53bc6431 LogTimestamp=2022-04-26T06:17:13.431662Z ResponseDesc=OK Exiting. 2022-04-26T06:17:13.434Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc/v1/artifactTypes HTTP/1.1 SC="200" 2022-04-26T06:17:13.609Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=90282a2b-ab02-4392-9ec7-cac3d993baa6 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=aai ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.67.16 StatusCode=INPROGRESS RemoteHost=10.233.67.16 ServiceName=v1/registerForDistribution ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:13.608855Z InvocationID=0469e3a2-e4b3-44af-b6d3-bf76a4a58b65 LogTimestamp=2022-04-26T06:17:13.609001Z Entering 2022-04-26T06:17:13.610Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=90282a2b-ab02-4392-9ec7-cac3d993baa6 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=aai ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.67.16 StatusCode=INPROGRESS userId=null RemoteHost=10.233.67.16 ServiceName=v1/registerForDistribution ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:13.608855Z InvocationID=0469e3a2-e4b3-44af-b6d3-bf76a4a58b65 LogTimestamp=2022-04-26T06:17:13.609001Z localAddr=10.233.65.41 remoteAddr=10.233.67.16 POST /sdc/v1/registerForDistribution HTTP/1.1 2022-04-26T06:17:13.628Z [qtp2085002312-20] WARN c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:13.610Z InvocationID=a123f412-7a66-4661-9649-c102330602c7 PartnerName=Apache-HttpClient/4.5.10 (Java/11.0.9) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/TSkjLZF9fnYJyEOK will send credentials over a clear channel. 2022-04-26T06:17:13.629Z [qtp2085002312-20] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:13.610Z InvocationID=a123f412-7a66-4661-9649-c102330602c7 PartnerName=Apache-HttpClient/4.5.10 (Java/11.0.9) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/TSkjLZF9fnYJyEOK (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:17:13.659Z [qtp2085002312-20] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:13.610Z InvocationID=a123f412-7a66-4661-9649-c102330602c7 PartnerName=Apache-HttpClient/4.5.10 (Java/11.0.9) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb \t--> HTTP/1.1 200 OK 2022-04-26T06:17:13.845Z [qtp2085002312-20] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "TSkjLZF9fnYJyEOK" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:13.610Z InvocationID=a123f412-7a66-4661-9649-c102330602c7 PartnerName=Apache-HttpClient/4.5.10 (Java/11.0.9) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/TSkjLZF9fnYJyEOK will send credentials over a clear channel. 2022-04-26T06:17:13.845Z [qtp2085002312-20] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "TSkjLZF9fnYJyEOK" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:13.610Z InvocationID=a123f412-7a66-4661-9649-c102330602c7 PartnerName=Apache-HttpClient/4.5.10 (Java/11.0.9) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/TSkjLZF9fnYJyEOK (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:17:13.870Z [qtp2085002312-20] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "TSkjLZF9fnYJyEOK" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:13.610Z InvocationID=a123f412-7a66-4661-9649-c102330602c7 PartnerName=Apache-HttpClient/4.5.10 (Java/11.0.9) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb \t--> HTTP/1.1 200 OK 2022-04-26T06:17:13.883Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "DRegister" CONSUMER_ID = "aai-ml" API_KEY = "TSkjLZF9fnYJyEOK" D_ENV = "AUTO" STATUS = "200" DESC = "" DNOTIF_TOPIC = "SDC-DISTR-NOTIF-TOPIC-AUTO" DSTATUS_TOPIC = "SDC-DISTR-STATUS-TOPIC-AUTO" ResponseCode=200 RequestID=null ElapsedTime=272 PartnerName=Apache-HttpClient/4.5.10 (Java/11.0.9) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:13.610Z InvocationID=a123f412-7a66-4661-9649-c102330602c7 LogTimestamp=2022-04-26T06:17:13.882776Z ResponseDesc=OK Exiting. 2022-04-26T06:17:13.885Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc/v1/registerForDistribution HTTP/1.1 SC="200" 2022-04-26T06:17:14.237Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2cdb8e9-9120-42fb-a7c2-a6844b23a332 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:17:14.238Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2cdb8e9-9120-42fb-a7c2-a6844b23a332 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5d420c67 2022-04-26T06:17:14.238Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2cdb8e9-9120-42fb-a7c2-a6844b23a332 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:17:14.238Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2cdb8e9-9120-42fb-a7c2-a6844b23a332 PartnerName=SDC.BE The cassandra down nodes number is 0 2022-04-26T06:17:17.251Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e2cdb8e9-9120-42fb-a7c2-a6844b23a332 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:17:17.251Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e2cdb8e9-9120-42fb-a7c2-a6844b23a332 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:17:17.252Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=e2cdb8e9-9120-42fb-a7c2-a6844b23a332 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:17:17.252Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=e2cdb8e9-9120-42fb-a7c2-a6844b23a332 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:17:20.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:17:20.380Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:17:22.766Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=da2cbd19-9baf-4b34-b98b-44ce2130ab4b InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=dcae auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.67.47 StatusCode=INPROGRESS RemoteHost=10.233.67.47 ServerIPAddress=10.233.65.41 ServiceName=v1/artifactTypes ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:22.766153Z InvocationID=2367963a-1e78-4e36-af5d-44cec2ef004d LogTimestamp=2022-04-26T06:17:22.766366Z Entering 2022-04-26T06:17:22.767Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=da2cbd19-9baf-4b34-b98b-44ce2130ab4b InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=dcae auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.67.47 StatusCode=INPROGRESS userId=null RemoteHost=10.233.67.47 ServerIPAddress=10.233.65.41 ServiceName=v1/artifactTypes ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:22.766153Z InvocationID=2367963a-1e78-4e36-af5d-44cec2ef004d LogTimestamp=2022-04-26T06:17:22.766366Z localAddr=10.233.65.41 remoteAddr=10.233.67.47 GET /sdc/v1/artifactTypes HTTP/1.1 2022-04-26T06:17:22.770Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=1 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/artifactTypes EntryTimestamp=2022-04-26T06:17:22.768Z InvocationID=c7595c77-dab9-4b2a-b6d1-bf4bcce8c1aa LogTimestamp=2022-04-26T06:17:22.769741Z ResponseDesc=OK Exiting. 2022-04-26T06:17:22.771Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc/v1/artifactTypes HTTP/1.1 SC="200" 2022-04-26T06:17:22.926Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=8e693547-b110-48dc-821f-0fbc96065440 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=dcae auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.67.47 StatusCode=INPROGRESS RemoteHost=10.233.67.47 ServerIPAddress=10.233.65.41 ServiceName=v1/registerForDistribution ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:22.926133Z InvocationID=6f7b4c74-da43-45e7-ba49-f51bf181e3a4 LogTimestamp=2022-04-26T06:17:22.926297Z Entering 2022-04-26T06:17:22.927Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=8e693547-b110-48dc-821f-0fbc96065440 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=dcae auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.67.47 StatusCode=INPROGRESS userId=null RemoteHost=10.233.67.47 ServerIPAddress=10.233.65.41 ServiceName=v1/registerForDistribution ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:22.926133Z InvocationID=6f7b4c74-da43-45e7-ba49-f51bf181e3a4 LogTimestamp=2022-04-26T06:17:22.926297Z localAddr=10.233.65.41 remoteAddr=10.233.67.47 POST /sdc/v1/registerForDistribution HTTP/1.1 2022-04-26T06:17:22.934Z [qtp2085002312-13] WARN c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:22.927Z InvocationID=83366707-4fb4-47ec-926e-917636ed0491 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/k7kqMYctFSJTg6th will send credentials over a clear channel. 2022-04-26T06:17:22.934Z [qtp2085002312-13] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:22.927Z InvocationID=83366707-4fb4-47ec-926e-917636ed0491 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/k7kqMYctFSJTg6th (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:17:22.959Z [qtp2085002312-13] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:22.927Z InvocationID=83366707-4fb4-47ec-926e-917636ed0491 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb \t--> HTTP/1.1 200 OK 2022-04-26T06:17:22.968Z [qtp2085002312-13] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "k7kqMYctFSJTg6th" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:22.927Z InvocationID=83366707-4fb4-47ec-926e-917636ed0491 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/k7kqMYctFSJTg6th will send credentials over a clear channel. 2022-04-26T06:17:22.969Z [qtp2085002312-13] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "k7kqMYctFSJTg6th" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:22.927Z InvocationID=83366707-4fb4-47ec-926e-917636ed0491 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/k7kqMYctFSJTg6th (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:17:22.995Z [qtp2085002312-13] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "k7kqMYctFSJTg6th" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:22.927Z InvocationID=83366707-4fb4-47ec-926e-917636ed0491 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb \t--> HTTP/1.1 200 OK 2022-04-26T06:17:23.056Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "DRegister" CONSUMER_ID = "ves-openapi-manager" API_KEY = "k7kqMYctFSJTg6th" D_ENV = "AUTO" STATUS = "200" DESC = "" DNOTIF_TOPIC = "SDC-DISTR-NOTIF-TOPIC-AUTO" DSTATUS_TOPIC = "SDC-DISTR-STATUS-TOPIC-AUTO" ResponseCode=200 RequestID=null ElapsedTime=128 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:22.927Z InvocationID=83366707-4fb4-47ec-926e-917636ed0491 LogTimestamp=2022-04-26T06:17:23.055589Z ResponseDesc=OK Exiting. 2022-04-26T06:17:23.057Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc/v1/registerForDistribution HTTP/1.1 SC="200" 2022-04-26T06:17:25.215Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=9763af06-199a-4b32-b522-ecc95c6ec25e InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=multicloud ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.180 StatusCode=INPROGRESS RemoteHost=10.233.69.180 ServiceName=v1/artifactTypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:25.214666Z InvocationID=fa66ca77-9430-4fe5-9020-73f050a5a965 LogTimestamp=2022-04-26T06:17:25.214839Z Entering 2022-04-26T06:17:25.215Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=9763af06-199a-4b32-b522-ecc95c6ec25e InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=multicloud ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.180 StatusCode=INPROGRESS userId=null RemoteHost=10.233.69.180 ServiceName=v1/artifactTypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:25.214666Z InvocationID=fa66ca77-9430-4fe5-9020-73f050a5a965 LogTimestamp=2022-04-26T06:17:25.214839Z localAddr=10.233.65.41 remoteAddr=10.233.69.180 GET /sdc/v1/artifactTypes HTTP/1.1 2022-04-26T06:17:25.218Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=1 PartnerName=Apache-HttpClient/4.5.7 (Java/11.0.11) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/artifactTypes EntryTimestamp=2022-04-26T06:17:25.216Z InvocationID=6cb1a898-5875-4866-8a49-c05d62c74924 LogTimestamp=2022-04-26T06:17:25.217871Z ResponseDesc=OK Exiting. 2022-04-26T06:17:25.219Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc/v1/artifactTypes HTTP/1.1 SC="200" 2022-04-26T06:17:25.318Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=b1a2f920-d61e-4874-9969-1e780659154e InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=multicloud ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.180 StatusCode=INPROGRESS RemoteHost=10.233.69.180 ServiceName=v1/registerForDistribution ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:25.318105Z InvocationID=722640b9-d333-4ddf-a9f8-5244a967b722 LogTimestamp=2022-04-26T06:17:25.318317Z Entering 2022-04-26T06:17:25.319Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=b1a2f920-d61e-4874-9969-1e780659154e InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=multicloud ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.180 StatusCode=INPROGRESS userId=null RemoteHost=10.233.69.180 ServiceName=v1/registerForDistribution ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:25.318105Z InvocationID=722640b9-d333-4ddf-a9f8-5244a967b722 LogTimestamp=2022-04-26T06:17:25.318317Z localAddr=10.233.65.41 remoteAddr=10.233.69.180 POST /sdc/v1/registerForDistribution HTTP/1.1 2022-04-26T06:17:25.327Z [qtp2085002312-13] WARN c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:25.320Z InvocationID=d5051b28-59cf-4d68-9ae8-dad52fd23038 PartnerName=Apache-HttpClient/4.5.7 (Java/11.0.11) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/tNLJwL35ODojW5YP will send credentials over a clear channel. 2022-04-26T06:17:25.327Z [qtp2085002312-13] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:25.320Z InvocationID=d5051b28-59cf-4d68-9ae8-dad52fd23038 PartnerName=Apache-HttpClient/4.5.7 (Java/11.0.11) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/tNLJwL35ODojW5YP (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:17:25.346Z [qtp2085002312-13] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:25.320Z InvocationID=d5051b28-59cf-4d68-9ae8-dad52fd23038 PartnerName=Apache-HttpClient/4.5.7 (Java/11.0.11) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb \t--> HTTP/1.1 200 OK 2022-04-26T06:17:25.353Z [qtp2085002312-13] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "tNLJwL35ODojW5YP" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:25.320Z InvocationID=d5051b28-59cf-4d68-9ae8-dad52fd23038 PartnerName=Apache-HttpClient/4.5.7 (Java/11.0.11) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/tNLJwL35ODojW5YP will send credentials over a clear channel. 2022-04-26T06:17:25.354Z [qtp2085002312-13] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "tNLJwL35ODojW5YP" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:25.320Z InvocationID=d5051b28-59cf-4d68-9ae8-dad52fd23038 PartnerName=Apache-HttpClient/4.5.7 (Java/11.0.11) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/tNLJwL35ODojW5YP (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:17:25.383Z [qtp2085002312-13] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "tNLJwL35ODojW5YP" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:25.320Z InvocationID=d5051b28-59cf-4d68-9ae8-dad52fd23038 PartnerName=Apache-HttpClient/4.5.7 (Java/11.0.11) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb \t--> HTTP/1.1 200 OK 2022-04-26T06:17:25.454Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "DRegister" CONSUMER_ID = "multicloud-k8s-id" API_KEY = "tNLJwL35ODojW5YP" D_ENV = "AUTO" STATUS = "200" DESC = "" DNOTIF_TOPIC = "SDC-DISTR-NOTIF-TOPIC-AUTO" DSTATUS_TOPIC = "SDC-DISTR-STATUS-TOPIC-AUTO" ResponseCode=200 RequestID=null ElapsedTime=134 PartnerName=Apache-HttpClient/4.5.7 (Java/11.0.11) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:25.320Z InvocationID=d5051b28-59cf-4d68-9ae8-dad52fd23038 LogTimestamp=2022-04-26T06:17:25.454527Z ResponseDesc=OK Exiting. 2022-04-26T06:17:25.455Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc/v1/registerForDistribution HTTP/1.1 SC="200" 2022-04-26T06:17:25.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59601c64-20bb-4a32-94f8-0920a8326a34 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:17:25.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59601c64-20bb-4a32-94f8-0920a8326a34 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@376fb0ee 2022-04-26T06:17:25.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59601c64-20bb-4a32-94f8-0920a8326a34 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:17:25.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59601c64-20bb-4a32-94f8-0920a8326a34 PartnerName=SDC.BE The cassandra down nodes number is 0 2022-04-26T06:17:28.212Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=59601c64-20bb-4a32-94f8-0920a8326a34 ErrorCategory=INFO ServiceName=BE Component DE state changed from UNKNOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component DE state changed from UNKNOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:17:28.213Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=59601c64-20bb-4a32-94f8-0920a8326a34 ErrorCategory=INFO ServiceName=BE Component DE state changed from UNKNOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:17:32.369Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=b1261b6e-2281-48d3-a9cc-8198c679dedb InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=policy ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.213 StatusCode=INPROGRESS RemoteHost=10.233.66.213 ServiceName=v1/artifactTypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:32.368804Z InvocationID=577dcf00-8eaa-4c42-9e10-39f9e69f8cc7 LogTimestamp=2022-04-26T06:17:32.368970Z Entering 2022-04-26T06:17:32.369Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=b1261b6e-2281-48d3-a9cc-8198c679dedb InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=policy ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.213 StatusCode=INPROGRESS userId=null RemoteHost=10.233.66.213 ServiceName=v1/artifactTypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:32.368804Z InvocationID=577dcf00-8eaa-4c42-9e10-39f9e69f8cc7 LogTimestamp=2022-04-26T06:17:32.368970Z localAddr=10.233.65.41 remoteAddr=10.233.66.213 GET /sdc/v1/artifactTypes HTTP/1.1 2022-04-26T06:17:32.371Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=1 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.11) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/artifactTypes EntryTimestamp=2022-04-26T06:17:32.370Z InvocationID=e0561f56-3009-42a8-a3ec-be6ed227744c LogTimestamp=2022-04-26T06:17:32.371543Z ResponseDesc=OK Exiting. 2022-04-26T06:17:32.372Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc/v1/artifactTypes HTTP/1.1 SC="200" 2022-04-26T06:17:32.598Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=96b6ee42-27d9-41f4-b108-3c5bd54c0ff4 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=policy ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.213 StatusCode=INPROGRESS RemoteHost=10.233.66.213 ServiceName=v1/registerForDistribution ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:32.598483Z InvocationID=1a1c0ee7-7bd9-4988-9cca-cbfc2db2a2bc LogTimestamp=2022-04-26T06:17:32.598592Z Entering 2022-04-26T06:17:32.599Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=96b6ee42-27d9-41f4-b108-3c5bd54c0ff4 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=policy ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.213 StatusCode=INPROGRESS userId=null RemoteHost=10.233.66.213 ServiceName=v1/registerForDistribution ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:32.598483Z InvocationID=1a1c0ee7-7bd9-4988-9cca-cbfc2db2a2bc LogTimestamp=2022-04-26T06:17:32.598592Z localAddr=10.233.65.41 remoteAddr=10.233.66.213 POST /sdc/v1/registerForDistribution HTTP/1.1 2022-04-26T06:17:32.603Z [qtp2085002312-15] WARN c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:32.599Z InvocationID=51c32bb4-4f29-4183-8e91-803b87193535 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.11) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/x9M3feUytsowDRQC will send credentials over a clear channel. 2022-04-26T06:17:32.603Z [qtp2085002312-15] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:32.599Z InvocationID=51c32bb4-4f29-4183-8e91-803b87193535 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.11) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/x9M3feUytsowDRQC (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:17:32.623Z [qtp2085002312-15] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:32.599Z InvocationID=51c32bb4-4f29-4183-8e91-803b87193535 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.11) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb \t--> HTTP/1.1 200 OK 2022-04-26T06:17:32.643Z [qtp2085002312-15] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "x9M3feUytsowDRQC" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:32.599Z InvocationID=51c32bb4-4f29-4183-8e91-803b87193535 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.11) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/x9M3feUytsowDRQC will send credentials over a clear channel. 2022-04-26T06:17:32.643Z [qtp2085002312-15] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "x9M3feUytsowDRQC" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:32.599Z InvocationID=51c32bb4-4f29-4183-8e91-803b87193535 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.11) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/x9M3feUytsowDRQC (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:17:32.661Z [qtp2085002312-15] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "x9M3feUytsowDRQC" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:32.599Z InvocationID=51c32bb4-4f29-4183-8e91-803b87193535 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.11) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb \t--> HTTP/1.1 200 OK 2022-04-26T06:17:32.944Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "DRegister" CONSUMER_ID = "policy-id" API_KEY = "x9M3feUytsowDRQC" D_ENV = "AUTO" STATUS = "200" DESC = "" DNOTIF_TOPIC = "SDC-DISTR-NOTIF-TOPIC-AUTO" DSTATUS_TOPIC = "SDC-DISTR-STATUS-TOPIC-AUTO" ResponseCode=200 RequestID=null ElapsedTime=345 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.11) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:32.599Z InvocationID=51c32bb4-4f29-4183-8e91-803b87193535 LogTimestamp=2022-04-26T06:17:32.944308Z ResponseDesc=OK Exiting. 2022-04-26T06:17:32.946Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc/v1/registerForDistribution HTTP/1.1 SC="200" 2022-04-26T06:17:34.846Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d80ec016-b6df-41cc-b4dc-51886be0454f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:17:34.846Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d80ec016-b6df-41cc-b4dc-51886be0454f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3b45b246 2022-04-26T06:17:34.847Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d80ec016-b6df-41cc-b4dc-51886be0454f PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:17:34.847Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d80ec016-b6df-41cc-b4dc-51886be0454f PartnerName=SDC.BE The cassandra down nodes number is 0 2022-04-26T06:17:35.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:17:35.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:17:37.834Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1c063b53-56af-4640-ae7e-4a79355383ac InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=sdnc ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.67.108 StatusCode=INPROGRESS RemoteHost=10.233.67.108 ServiceName=v1/artifactTypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:37.834094Z InvocationID=734268f3-6a8c-4bc5-abfe-6a75ab996bdd LogTimestamp=2022-04-26T06:17:37.834304Z Entering 2022-04-26T06:17:37.835Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1c063b53-56af-4640-ae7e-4a79355383ac InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=sdnc ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.67.108 StatusCode=INPROGRESS userId=null RemoteHost=10.233.67.108 ServiceName=v1/artifactTypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:37.834094Z InvocationID=734268f3-6a8c-4bc5-abfe-6a75ab996bdd LogTimestamp=2022-04-26T06:17:37.834304Z localAddr=10.233.65.41 remoteAddr=10.233.67.108 GET /sdc/v1/artifactTypes HTTP/1.1 2022-04-26T06:17:37.836Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=1 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/artifactTypes EntryTimestamp=2022-04-26T06:17:37.835Z InvocationID=56ddbfab-ce6a-4338-ad78-237c181d7ca8 LogTimestamp=2022-04-26T06:17:37.836663Z ResponseDesc=OK Exiting. 2022-04-26T06:17:37.838Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc/v1/artifactTypes HTTP/1.1 SC="200" 2022-04-26T06:17:37.968Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=cf0b8e27-53a2-485d-ab83-ee38e2ad0116 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=sdnc ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.67.108 StatusCode=INPROGRESS RemoteHost=10.233.67.108 ServiceName=v1/registerForDistribution ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:37.967668Z InvocationID=7cdf97f8-f017-4cbe-a6d8-7884c933199f LogTimestamp=2022-04-26T06:17:37.967811Z Entering 2022-04-26T06:17:37.968Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=cf0b8e27-53a2-485d-ab83-ee38e2ad0116 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=sdnc ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.67.108 StatusCode=INPROGRESS userId=null RemoteHost=10.233.67.108 ServiceName=v1/registerForDistribution ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:37.967668Z InvocationID=7cdf97f8-f017-4cbe-a6d8-7884c933199f LogTimestamp=2022-04-26T06:17:37.967811Z localAddr=10.233.65.41 remoteAddr=10.233.67.108 POST /sdc/v1/registerForDistribution HTTP/1.1 2022-04-26T06:17:37.972Z [qtp2085002312-15] WARN c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:37.968Z InvocationID=8b8b67bb-c558-4cad-8a62-a8f08289cfd5 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/VfMp3k0Uuc5lLCkZ will send credentials over a clear channel. 2022-04-26T06:17:37.973Z [qtp2085002312-15] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:37.968Z InvocationID=8b8b67bb-c558-4cad-8a62-a8f08289cfd5 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/VfMp3k0Uuc5lLCkZ (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:17:37.995Z [qtp2085002312-15] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:37.968Z InvocationID=8b8b67bb-c558-4cad-8a62-a8f08289cfd5 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb \t--> HTTP/1.1 200 OK 2022-04-26T06:17:38.044Z [qtp2085002312-15] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "VfMp3k0Uuc5lLCkZ" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:37.968Z InvocationID=8b8b67bb-c558-4cad-8a62-a8f08289cfd5 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/VfMp3k0Uuc5lLCkZ will send credentials over a clear channel. 2022-04-26T06:17:38.045Z [qtp2085002312-15] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "VfMp3k0Uuc5lLCkZ" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:37.968Z InvocationID=8b8b67bb-c558-4cad-8a62-a8f08289cfd5 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/VfMp3k0Uuc5lLCkZ (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:17:38.066Z [qtp2085002312-15] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "VfMp3k0Uuc5lLCkZ" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:37.968Z InvocationID=8b8b67bb-c558-4cad-8a62-a8f08289cfd5 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb \t--> HTTP/1.1 200 OK 2022-04-26T06:17:38.077Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "DRegister" CONSUMER_ID = "sdc-COpenSource-Env11-sdnc-dockero" API_KEY = "VfMp3k0Uuc5lLCkZ" D_ENV = "AUTO" STATUS = "200" DESC = "" DNOTIF_TOPIC = "SDC-DISTR-NOTIF-TOPIC-AUTO" DSTATUS_TOPIC = "SDC-DISTR-STATUS-TOPIC-AUTO" ResponseCode=200 RequestID=null ElapsedTime=108 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:37.968Z InvocationID=8b8b67bb-c558-4cad-8a62-a8f08289cfd5 LogTimestamp=2022-04-26T06:17:38.076895Z ResponseDesc=OK Exiting. 2022-04-26T06:17:38.078Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc/v1/registerForDistribution HTTP/1.1 SC="200" 2022-04-26T06:17:40.359Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=e4f70030-3e9d-4ddb-8687-00e770b220f2 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=cds ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.142 StatusCode=INPROGRESS RemoteHost=10.233.69.142 ServiceName=v1/artifactTypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:40.358454Z InvocationID=2c5d4057-d4f3-432a-9ebb-5d124dbf05a2 LogTimestamp=2022-04-26T06:17:40.358617Z Entering 2022-04-26T06:17:40.359Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=e4f70030-3e9d-4ddb-8687-00e770b220f2 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=cds ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.142 StatusCode=INPROGRESS userId=null RemoteHost=10.233.69.142 ServiceName=v1/artifactTypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:40.358454Z InvocationID=2c5d4057-d4f3-432a-9ebb-5d124dbf05a2 LogTimestamp=2022-04-26T06:17:40.358617Z localAddr=10.233.65.41 remoteAddr=10.233.69.142 GET /sdc/v1/artifactTypes HTTP/1.1 2022-04-26T06:17:40.362Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=2 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/artifactTypes EntryTimestamp=2022-04-26T06:17:40.360Z InvocationID=633408c3-38e0-4d77-8068-5030f95ba30a LogTimestamp=2022-04-26T06:17:40.362108Z ResponseDesc=OK Exiting. 2022-04-26T06:17:40.363Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc/v1/artifactTypes HTTP/1.1 SC="200" 2022-04-26T06:17:40.478Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=25bf9709-5cab-477c-8256-c93c777eda54 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=cds ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.142 StatusCode=INPROGRESS RemoteHost=10.233.69.142 ServiceName=v1/registerForDistribution ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:40.477523Z InvocationID=42406d3e-9201-4a47-9e0a-983a8973e5f8 LogTimestamp=2022-04-26T06:17:40.477688Z Entering 2022-04-26T06:17:40.479Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=25bf9709-5cab-477c-8256-c93c777eda54 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=cds ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.142 StatusCode=INPROGRESS userId=null RemoteHost=10.233.69.142 ServiceName=v1/registerForDistribution ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:40.477523Z InvocationID=42406d3e-9201-4a47-9e0a-983a8973e5f8 LogTimestamp=2022-04-26T06:17:40.477688Z localAddr=10.233.65.41 remoteAddr=10.233.69.142 POST /sdc/v1/registerForDistribution HTTP/1.1 2022-04-26T06:17:40.484Z [qtp2085002312-15] WARN c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:40.479Z InvocationID=a96d2c0c-e267-4a54-95e8-df32ec238e58 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/Sgub3E8Vjr4HEbEJ will send credentials over a clear channel. 2022-04-26T06:17:40.484Z [qtp2085002312-15] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:40.479Z InvocationID=a96d2c0c-e267-4a54-95e8-df32ec238e58 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/Sgub3E8Vjr4HEbEJ (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:17:40.503Z [qtp2085002312-15] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:40.479Z InvocationID=a96d2c0c-e267-4a54-95e8-df32ec238e58 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb \t--> HTTP/1.1 200 OK 2022-04-26T06:17:40.509Z [qtp2085002312-15] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "Sgub3E8Vjr4HEbEJ" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:40.479Z InvocationID=a96d2c0c-e267-4a54-95e8-df32ec238e58 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/Sgub3E8Vjr4HEbEJ will send credentials over a clear channel. 2022-04-26T06:17:40.510Z [qtp2085002312-15] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "Sgub3E8Vjr4HEbEJ" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:40.479Z InvocationID=a96d2c0c-e267-4a54-95e8-df32ec238e58 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/Sgub3E8Vjr4HEbEJ (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:17:40.530Z [qtp2085002312-15] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "Sgub3E8Vjr4HEbEJ" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:40.479Z InvocationID=a96d2c0c-e267-4a54-95e8-df32ec238e58 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb \t--> HTTP/1.1 200 OK 2022-04-26T06:17:40.646Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "DRegister" CONSUMER_ID = "cds" API_KEY = "Sgub3E8Vjr4HEbEJ" D_ENV = "AUTO" STATUS = "200" DESC = "" DNOTIF_TOPIC = "SDC-DISTR-NOTIF-TOPIC-AUTO" DSTATUS_TOPIC = "SDC-DISTR-STATUS-TOPIC-AUTO" ResponseCode=200 RequestID=null ElapsedTime=166 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:40.479Z InvocationID=a96d2c0c-e267-4a54-95e8-df32ec238e58 LogTimestamp=2022-04-26T06:17:40.645902Z ResponseDesc=OK Exiting. 2022-04-26T06:17:40.647Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc/v1/registerForDistribution HTTP/1.1 SC="200" 2022-04-26T06:17:43.540Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e26c0f8-1be1-4972-a172-e80240988746 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:17:43.540Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e26c0f8-1be1-4972-a172-e80240988746 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@37059584 2022-04-26T06:17:43.540Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e26c0f8-1be1-4972-a172-e80240988746 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:17:43.540Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e26c0f8-1be1-4972-a172-e80240988746 PartnerName=SDC.BE The cassandra down nodes number is 0 2022-04-26T06:17:46.546Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=2e26c0f8-1be1-4972-a172-e80240988746 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:17:46.547Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=2e26c0f8-1be1-4972-a172-e80240988746 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:17:46.547Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=2e26c0f8-1be1-4972-a172-e80240988746 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:17:46.547Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=2e26c0f8-1be1-4972-a172-e80240988746 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:17:49.771Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=497c9374-bc54-4c48-9865-62e4a890f774 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/user/jh0003 ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:49.770857Z InvocationID=02f51aa6-1a9e-4e32-8d1a-b799469eab17 LogTimestamp=2022-04-26T06:17:49.770964Z Entering 2022-04-26T06:17:49.772Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=497c9374-bc54-4c48-9865-62e4a890f774 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/user/jh0003 ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:49.770857Z InvocationID=02f51aa6-1a9e-4e32-8d1a-b799469eab17 LogTimestamp=2022-04-26T06:17:49.770964Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2022-04-26T06:17:49.835Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=62 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/user/jh0003 EntryTimestamp=2022-04-26T06:17:49.772Z InvocationID=0d8b77c3-5a2c-4792-a227-43f2c12d67c7 LogTimestamp=2022-04-26T06:17:49.834921Z ResponseDesc=OK Exiting. 2022-04-26T06:17:49.836Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2022-04-26T06:17:49.874Z [qtp2085002312-13] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:17:49.874504Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:17:49.875Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:17:49.874504Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:17:49.876Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdemo HTTP/1.1 SC="500" 2022-04-26T06:17:49.891Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=4a5926d5-5e1e-4e4c-bb42-567c7167c40e InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/user ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:49.890516Z InvocationID=3bffd752-27c7-4860-8cd4-e03257b4562d LogTimestamp=2022-04-26T06:17:49.890666Z Entering 2022-04-26T06:17:49.891Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=4a5926d5-5e1e-4e4c-bb42-567c7167c40e InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/user ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:49.890516Z InvocationID=3bffd752-27c7-4860-8cd4-e03257b4562d LogTimestamp=2022-04-26T06:17:49.890666Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/user HTTP/1.1 2022-04-26T06:17:50.321Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f252cc99-0f83-4674-96da-25aca5995f51 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=mso ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.16 StatusCode=INPROGRESS RemoteHost=10.233.69.16 ServiceName=v1/artifactTypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:50.321423Z InvocationID=21101dc9-1b66-42e8-9fbe-964b3461fd24 LogTimestamp=2022-04-26T06:17:50.321560Z Entering 2022-04-26T06:17:50.322Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f252cc99-0f83-4674-96da-25aca5995f51 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=mso ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.16 StatusCode=INPROGRESS userId=null RemoteHost=10.233.69.16 ServiceName=v1/artifactTypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:50.321423Z InvocationID=21101dc9-1b66-42e8-9fbe-964b3461fd24 LogTimestamp=2022-04-26T06:17:50.321560Z localAddr=10.233.65.41 remoteAddr=10.233.69.16 GET /sdc/v1/artifactTypes HTTP/1.1 2022-04-26T06:17:50.324Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=1 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/artifactTypes EntryTimestamp=2022-04-26T06:17:50.322Z InvocationID=bb6ec80d-8bff-4756-b6c6-9896feb1d9bc LogTimestamp=2022-04-26T06:17:50.323933Z ResponseDesc=OK Exiting. 2022-04-26T06:17:50.324Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc/v1/artifactTypes HTTP/1.1 SC="200" 2022-04-26T06:17:50.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:17:50.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:17:50.467Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a5047676-0271-42b7-9149-c8f3f07c8aa6 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=mso ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.16 StatusCode=INPROGRESS RemoteHost=10.233.69.16 ServiceName=v1/registerForDistribution ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:50.467096Z InvocationID=965d3f07-4d89-4693-9036-fb0c512eaf56 LogTimestamp=2022-04-26T06:17:50.467286Z Entering 2022-04-26T06:17:50.468Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a5047676-0271-42b7-9149-c8f3f07c8aa6 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=mso ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.16 StatusCode=INPROGRESS userId=null RemoteHost=10.233.69.16 ServiceName=v1/registerForDistribution ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:50.467096Z InvocationID=965d3f07-4d89-4693-9036-fb0c512eaf56 LogTimestamp=2022-04-26T06:17:50.467286Z localAddr=10.233.65.41 remoteAddr=10.233.69.16 POST /sdc/v1/registerForDistribution HTTP/1.1 2022-04-26T06:17:50.474Z [qtp2085002312-13] WARN c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:50.469Z InvocationID=df63a748-3649-4fa8-ab55-66a326bb1758 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/POwD3G6zHp5kVXCM will send credentials over a clear channel. 2022-04-26T06:17:50.474Z [qtp2085002312-13] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:50.469Z InvocationID=df63a748-3649-4fa8-ab55-66a326bb1758 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/POwD3G6zHp5kVXCM (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:17:50.498Z [qtp2085002312-13] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:50.469Z InvocationID=df63a748-3649-4fa8-ab55-66a326bb1758 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb \t--> HTTP/1.1 200 OK 2022-04-26T06:17:50.844Z [qtp2085002312-13] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "POwD3G6zHp5kVXCM" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:50.469Z InvocationID=df63a748-3649-4fa8-ab55-66a326bb1758 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/consumers/POwD3G6zHp5kVXCM will send credentials over a clear channel. 2022-04-26T06:17:50.844Z [qtp2085002312-13] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "POwD3G6zHp5kVXCM" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:50.469Z InvocationID=df63a748-3649-4fa8-ab55-66a326bb1758 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/consumers/POwD3G6zHp5kVXCM (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:17:50.869Z [qtp2085002312-13] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "POwD3G6zHp5kVXCM" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:50.469Z InvocationID=df63a748-3649-4fa8-ab55-66a326bb1758 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb \t--> HTTP/1.1 200 OK 2022-04-26T06:17:50.875Z [qtp2085002312-13] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "CONSUMER" API_KEY = "POwD3G6zHp5kVXCM" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:50.469Z InvocationID=df63a748-3649-4fa8-ab55-66a326bb1758 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/POwD3G6zHp5kVXCM will send credentials over a clear channel. 2022-04-26T06:17:50.875Z [qtp2085002312-13] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "CONSUMER" API_KEY = "POwD3G6zHp5kVXCM" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:50.469Z InvocationID=df63a748-3649-4fa8-ab55-66a326bb1758 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/POwD3G6zHp5kVXCM (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:17:50.893Z [qtp2085002312-13] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "CONSUMER" API_KEY = "POwD3G6zHp5kVXCM" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:50.469Z InvocationID=df63a748-3649-4fa8-ab55-66a326bb1758 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb \t--> HTTP/1.1 200 OK 2022-04-26T06:17:50.948Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "DRegister" CONSUMER_ID = "SO-COpenSource-Env11" API_KEY = "POwD3G6zHp5kVXCM" D_ENV = "AUTO" STATUS = "200" DESC = "" DNOTIF_TOPIC = "SDC-DISTR-NOTIF-TOPIC-AUTO" DSTATUS_TOPIC = "SDC-DISTR-STATUS-TOPIC-AUTO" ResponseCode=200 RequestID=null ElapsedTime=479 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:17:50.469Z InvocationID=df63a748-3649-4fa8-ab55-66a326bb1758 LogTimestamp=2022-04-26T06:17:50.948385Z ResponseDesc=OK Exiting. 2022-04-26T06:17:50.949Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc/v1/registerForDistribution HTTP/1.1 SC="200" 2022-04-26T06:17:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:17:52.686Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:17:52.686Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:17:54.040Z [qtp2085002312-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "demo demo demo demo@openecomp.org ADMIN" STATUS = "201" DESC = "OK" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/user EntryTimestamp=2022-04-26T06:17:49.892Z InvocationID=10d1c399-5057-4ed7-837d-655d2a03c29e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:17:54.042Z [qtp2085002312-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "demo demo demo demo@openecomp.org ADMIN" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/user TargetServiceName=null EntryTimestamp=2022-04-26T06:17:49.892Z InvocationID=10d1c399-5057-4ed7-837d-655d2a03c29e ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:17:54.045Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "demo demo demo demo@openecomp.org ADMIN" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=4152 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/user TargetServiceName=null EntryTimestamp=2022-04-26T06:17:49.892Z InvocationID=10d1c399-5057-4ed7-837d-655d2a03c29e ErrorCode=500 LogTimestamp=2022-04-26T06:17:54.044894Z ResponseDesc=Created Exiting. 2022-04-26T06:17:54.046Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="201" 2022-04-26T06:17:54.060Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=36188 PartnerName=kube-probe/1.22 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=ERROR ServerIPAddress=10.233.65.41 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2022-04-26T06:17:17.872Z InvocationID=232516f7-23ee-41af-a43d-43d79072fd30 ErrorCode=500 LogTimestamp=2022-04-26T06:17:54.060331Z ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:17:54.061Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usergv0001 HTTP/1.1 SC="500" 2022-04-26T06:17:54.071Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=49ff93e1-c70d-45c9-936e-bb9c9972a9b9 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:54.071516Z InvocationID=4394a8e3-dfae-47ea-88e4-2ff15ac2805d LogTimestamp=2022-04-26T06:17:54.071597Z Entering 2022-04-26T06:17:54.072Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=49ff93e1-c70d-45c9-936e-bb9c9972a9b9 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:54.071516Z InvocationID=4394a8e3-dfae-47ea-88e4-2ff15ac2805d LogTimestamp=2022-04-26T06:17:54.071597Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/user HTTP/1.1 2022-04-26T06:17:54.142Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c86f4a0e-e778-444d-a660-8fc4ff901c8b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:17:54.143Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c86f4a0e-e778-444d-a660-8fc4ff901c8b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@67e32408 2022-04-26T06:17:54.143Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c86f4a0e-e778-444d-a660-8fc4ff901c8b PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:17:54.143Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c86f4a0e-e778-444d-a660-8fc4ff901c8b PartnerName=SDC.BE The cassandra down nodes number is 0 2022-04-26T06:17:54.155Z [qtp2085002312-17] ERROR o.o.sdc.be.servlets.UserAdminServlet RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/user EntryTimestamp=2022-04-26T06:17:54.072Z InvocationID=5fb095ac-f37b-479d-b406-ca1a94db28ea PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@771974dc', 'User [firstName=Giuseppe, lastName=Verdi, userId=gv0001, email=gv0001@openecomp.org, role=GOVERNOR, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.user.UserBusinessLogic#validateRole[183] in 72ms 2022-04-26T06:17:54.160Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=400 RequestID=null ElapsedTime=88 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/user ErrorDesc=Bad Request EntryTimestamp=2022-04-26T06:17:54.072Z InvocationID=5fb095ac-f37b-479d-b406-ca1a94db28ea ErrorCode=400 LogTimestamp=2022-04-26T06:17:54.160052Z ResponseDesc=Bad Request Exiting. 2022-04-26T06:17:54.161Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="400" 2022-04-26T06:17:54.169Z [qtp2085002312-14] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:17:54.169554Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:17:54.169Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:17:54.169554Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:17:54.170Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjh0003 HTTP/1.1 SC="500" 2022-04-26T06:17:54.179Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=cffc8366-fd8d-45e5-b6d1-7859ba860910 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/user ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:54.178793Z InvocationID=ac68961b-5940-46f3-9b83-a5daf9e5da54 LogTimestamp=2022-04-26T06:17:54.178871Z Entering 2022-04-26T06:17:54.179Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=cffc8366-fd8d-45e5-b6d1-7859ba860910 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/user ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:54.178793Z InvocationID=ac68961b-5940-46f3-9b83-a5daf9e5da54 LogTimestamp=2022-04-26T06:17:54.178871Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/user HTTP/1.1 2022-04-26T06:17:54.196Z [qtp2085002312-20] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/user EntryTimestamp=2022-04-26T06:17:54.179Z InvocationID=772cc2a9-4221-4b1d-babd-93d7cf65e960 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@771974dc', 'User [firstName=Jimmy, lastName=Hendrix, userId=jh0003, email=jh0003@openecomp.org, role=Admin, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1119] in 16ms 2022-04-26T06:17:54.206Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." ResponseCode=409 RequestID=null ElapsedTime=27 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2022-04-26T06:17:54.179Z InvocationID=772cc2a9-4221-4b1d-babd-93d7cf65e960 ErrorCode=409 LogTimestamp=2022-04-26T06:17:54.206109Z ResponseDesc=Conflict Exiting. 2022-04-26T06:17:54.207Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2022-04-26T06:17:54.217Z [qtp2085002312-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:17:54.217287Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:17:54.217Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:17:54.217287Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:17:54.218Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjm0007 HTTP/1.1 SC="500" 2022-04-26T06:17:54.231Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=055d98b7-b125-401b-8464-43393359dabe InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/user ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:54.231241Z InvocationID=15adbdf7-b3af-4c13-a6f3-fbc6a72b60fe LogTimestamp=2022-04-26T06:17:54.231364Z Entering 2022-04-26T06:17:54.232Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=055d98b7-b125-401b-8464-43393359dabe InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/user ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:54.231241Z InvocationID=15adbdf7-b3af-4c13-a6f3-fbc6a72b60fe LogTimestamp=2022-04-26T06:17:54.231364Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/user HTTP/1.1 2022-04-26T06:17:56.257Z [qtp2085002312-13] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "jm0007 Joni Mitchell jm0007@openecomp.org TESTER" STATUS = "201" DESC = "OK" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/user EntryTimestamp=2022-04-26T06:17:54.232Z InvocationID=e539b324-4ad7-47a0-bb3d-2673e07c08e4 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:17:56.258Z [qtp2085002312-13] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "jm0007 Joni Mitchell jm0007@openecomp.org TESTER" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/user TargetServiceName=null EntryTimestamp=2022-04-26T06:17:54.232Z InvocationID=e539b324-4ad7-47a0-bb3d-2673e07c08e4 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:17:56.260Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "jm0007 Joni Mitchell jm0007@openecomp.org TESTER" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=2027 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/user TargetServiceName=null EntryTimestamp=2022-04-26T06:17:54.232Z InvocationID=e539b324-4ad7-47a0-bb3d-2673e07c08e4 ErrorCode=500 LogTimestamp=2022-04-26T06:17:56.259644Z ResponseDesc=Created Exiting. 2022-04-26T06:17:56.260Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="201" 2022-04-26T06:17:56.273Z [qtp2085002312-18] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:17:56.273563Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:17:56.274Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:17:56.273563Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:17:56.275Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercs0008 HTTP/1.1 SC="500" 2022-04-26T06:17:56.286Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=355b5ffa-ba18-42f3-8ca6-a3ccc02c4532 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/user ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:56.285735Z InvocationID=71bd5a70-53e6-4643-b77c-71b3980d3381 LogTimestamp=2022-04-26T06:17:56.285855Z Entering 2022-04-26T06:17:56.286Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=355b5ffa-ba18-42f3-8ca6-a3ccc02c4532 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/user ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:56.285735Z InvocationID=71bd5a70-53e6-4643-b77c-71b3980d3381 LogTimestamp=2022-04-26T06:17:56.285855Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/user HTTP/1.1 2022-04-26T06:17:56.860Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=c86f4a0e-e778-444d-a660-8fc4ff901c8b ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:17:56.860Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=c86f4a0e-e778-444d-a660-8fc4ff901c8b ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:17:57.350Z [qtp2085002312-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "cs0008 Carlos Santana cs0008r@openecomp.org DESIGNER" STATUS = "201" DESC = "OK" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/user EntryTimestamp=2022-04-26T06:17:56.286Z InvocationID=52eff004-d2d2-4d8a-82ed-ccbc9f43db67 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:17:57.351Z [qtp2085002312-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "cs0008 Carlos Santana cs0008r@openecomp.org DESIGNER" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/user TargetServiceName=null EntryTimestamp=2022-04-26T06:17:56.286Z InvocationID=52eff004-d2d2-4d8a-82ed-ccbc9f43db67 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:17:57.352Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "cs0008 Carlos Santana cs0008r@openecomp.org DESIGNER" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=1066 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/user TargetServiceName=null EntryTimestamp=2022-04-26T06:17:56.286Z InvocationID=52eff004-d2d2-4d8a-82ed-ccbc9f43db67 ErrorCode=500 LogTimestamp=2022-04-26T06:17:57.352009Z ResponseDesc=Created Exiting. 2022-04-26T06:17:57.353Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="201" 2022-04-26T06:17:57.549Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=28a269f3-919b-4d88-b980-a20dddea1038 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/user/jh0003 ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:57.549167Z InvocationID=09815388-e2cf-4a57-aa59-4941a38dd7f7 LogTimestamp=2022-04-26T06:17:57.549274Z Entering 2022-04-26T06:17:57.549Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=28a269f3-919b-4d88-b980-a20dddea1038 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/user/jh0003 ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:57.549167Z InvocationID=09815388-e2cf-4a57-aa59-4941a38dd7f7 LogTimestamp=2022-04-26T06:17:57.549274Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2022-04-26T06:17:57.555Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/user/jh0003 EntryTimestamp=2022-04-26T06:17:57.550Z InvocationID=ba2ab862-a122-4a86-94c2-aa0ff35d27ab LogTimestamp=2022-04-26T06:17:57.555381Z ResponseDesc=OK Exiting. 2022-04-26T06:17:57.556Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2022-04-26T06:17:57.566Z [qtp2085002312-18] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:17:57.565973Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:17:57.566Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:17:57.565973Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:17:57.567Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/useraai HTTP/1.1 SC="500" 2022-04-26T06:17:57.583Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0b2584fe-13fc-4855-a640-85b07c0c165e InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:57.583353Z InvocationID=92bef71c-f9e4-4ba5-a538-2608b9579542 LogTimestamp=2022-04-26T06:17:57.583445Z Entering 2022-04-26T06:17:57.584Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0b2584fe-13fc-4855-a640-85b07c0c165e InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2022-04-26T06:17:57.583353Z InvocationID=92bef71c-f9e4-4ba5-a538-2608b9579542 LogTimestamp=2022-04-26T06:17:57.583445Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:17:57.611Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:17:57.584Z InvocationID=e8d0e91e-d9da-40c1-87a5-187430b230d9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id aai 2022-04-26T06:18:01.650Z [qtp2085002312-12] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:17:57.584Z InvocationID=e8d0e91e-d9da-40c1-87a5-187430b230d9 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Error returned after get Consumer Data node aai. status returned is NOT_FOUND 2022-04-26T06:18:02.947Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5aef8772-1090-4865-82a3-d5975588c909 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:02.947Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5aef8772-1090-4865-82a3-d5975588c909 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@323a80da 2022-04-26T06:18:02.947Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5aef8772-1090-4865-82a3-d5975588c909 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:02.947Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5aef8772-1090-4865-82a3-d5975588c909 PartnerName=SDC.BE The cassandra down nodes number is 0 2022-04-26T06:18:05.368Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:18:05.377Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:18:07.645Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:18:08.089Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f1336816-d69a-4ba9-a0f3-104c661b5030 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=clamp ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.72.165 StatusCode=INPROGRESS RemoteHost=10.233.72.165 ServiceName=v1/artifactTypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:08.088181Z InvocationID=8453895a-3f6f-44a2-9264-a01450291a3b LogTimestamp=2022-04-26T06:18:08.088740Z Entering 2022-04-26T06:18:08.089Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f1336816-d69a-4ba9-a0f3-104c661b5030 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=clamp ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.72.165 StatusCode=INPROGRESS userId=null RemoteHost=10.233.72.165 ServiceName=v1/artifactTypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:08.088181Z InvocationID=8453895a-3f6f-44a2-9264-a01450291a3b LogTimestamp=2022-04-26T06:18:08.088740Z localAddr=10.233.65.41 remoteAddr=10.233.72.165 GET /sdc/v1/artifactTypes HTTP/1.1 2022-04-26T06:18:08.093Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=3 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/artifactTypes EntryTimestamp=2022-04-26T06:18:08.090Z InvocationID=9885210d-7ce8-4cbe-9716-f057603b4fd7 LogTimestamp=2022-04-26T06:18:08.093331Z ResponseDesc=OK Exiting. 2022-04-26T06:18:08.094Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc/v1/artifactTypes HTTP/1.1 SC="200" 2022-04-26T06:18:08.257Z [qtp2085002312-12] ERROR o.o.s.b.d.cassandra.CassandraClient AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "aai,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:17:57.584Z InvocationID=e8d0e91e-d9da-40c1-87a5-187430b230d9 ErrorCode=300 Failed to save entity [ConsumerEvent(timebaseduuid=a4016000-c528-11ec-814b-87bf26af1973, timestamp1=Tue Apr 26 06:18:08 GMT 2022, requestId=0b2584fe-13fc-4855-a640-85b07c0c165e, action=AddECOMPUserCredentials, status=201, desc=OK, modifier=Jimmy Hendrix(jh0003), ecompUser=aai,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495)], error : java.lang.IllegalArgumentException: Table or materialized view consumerevent does not exist in keyspace sdcaudit at com.datastax.driver.mapping.AnnotationParser.parseEntity(AnnotationParser.java:103) at com.datastax.driver.mapping.MappingManager.getMapper(MappingManager.java:329) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:261) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:273) at org.openecomp.sdc.be.dao.cassandra.CassandraClient.save(CassandraClient.java:156) at org.openecomp.sdc.be.dao.cassandra.AuditCassandraDao.saveRecord(AuditCassandraDao.java:77) at org.openecomp.sdc.be.auditing.impl.AuditingManager.saveEventToCassandra(AuditingManager.java:74) at org.openecomp.sdc.be.auditing.impl.AuditingManager.auditEvent(AuditingManager.java:57) at org.openecomp.sdc.be.impl.ComponentsUtils.auditConsumerCredentialsEvent(ComponentsUtils.java:1105) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:108) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:18:08.283Z [qtp2085002312-12] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "aai,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:17:57.584Z InvocationID=e8d0e91e-d9da-40c1-87a5-187430b230d9 ErrorCode=400 Failed to persist to cassandra auditing event: GENERAL_ERROR 2022-04-26T06:18:08.310Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=aa01cd60-936d-474b-93bc-ca7e0a0b470e InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=clamp ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.72.165 StatusCode=INPROGRESS RemoteHost=10.233.72.165 ServiceName=v1/registerForDistribution ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:08.310371Z InvocationID=86be02bf-5767-40d4-a5b5-571ed9f0e8ce LogTimestamp=2022-04-26T06:18:08.310519Z Entering 2022-04-26T06:18:08.311Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=aa01cd60-936d-474b-93bc-ca7e0a0b470e InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=clamp ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.72.165 StatusCode=INPROGRESS userId=null RemoteHost=10.233.72.165 ServiceName=v1/registerForDistribution ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:08.310371Z InvocationID=86be02bf-5767-40d4-a5b5-571ed9f0e8ce LogTimestamp=2022-04-26T06:18:08.310519Z localAddr=10.233.65.41 remoteAddr=10.233.72.165 POST /sdc/v1/registerForDistribution HTTP/1.1 2022-04-26T06:18:08.318Z [qtp2085002312-14] WARN c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:18:08.311Z InvocationID=0bc25699-387c-4ff6-b24b-1c3a8576b4d9 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/uV2CqvYEnLP25KDw will send credentials over a clear channel. 2022-04-26T06:18:08.318Z [qtp2085002312-14] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:18:08.311Z InvocationID=0bc25699-387c-4ff6-b24b-1c3a8576b4d9 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/uV2CqvYEnLP25KDw (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:18:08.336Z [qtp2085002312-14] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:18:08.311Z InvocationID=0bc25699-387c-4ff6-b24b-1c3a8576b4d9 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb \t--> HTTP/1.1 200 OK 2022-04-26T06:18:08.345Z [qtp2085002312-14] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "uV2CqvYEnLP25KDw" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:18:08.311Z InvocationID=0bc25699-387c-4ff6-b24b-1c3a8576b4d9 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/uV2CqvYEnLP25KDw will send credentials over a clear channel. 2022-04-26T06:18:08.346Z [qtp2085002312-14] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "uV2CqvYEnLP25KDw" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:18:08.311Z InvocationID=0bc25699-387c-4ff6-b24b-1c3a8576b4d9 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/uV2CqvYEnLP25KDw (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:18:08.368Z [qtp2085002312-14] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "uV2CqvYEnLP25KDw" STATUS = "200" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:18:08.311Z InvocationID=0bc25699-387c-4ff6-b24b-1c3a8576b4d9 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb \t--> HTTP/1.1 200 OK 2022-04-26T06:18:08.546Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "DRegister" CONSUMER_ID = "clamp" API_KEY = "uV2CqvYEnLP25KDw" D_ENV = "AUTO" STATUS = "200" DESC = "" DNOTIF_TOPIC = "SDC-DISTR-NOTIF-TOPIC-AUTO" DSTATUS_TOPIC = "SDC-DISTR-STATUS-TOPIC-AUTO" ResponseCode=200 RequestID=null ElapsedTime=235 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/registerForDistribution EntryTimestamp=2022-04-26T06:18:08.311Z InvocationID=0bc25699-387c-4ff6-b24b-1c3a8576b4d9 LogTimestamp=2022-04-26T06:18:08.546001Z ResponseDesc=OK Exiting. 2022-04-26T06:18:08.547Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc/v1/registerForDistribution HTTP/1.1 SC="200" 2022-04-26T06:18:09.047Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "aai,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=11463 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:17:57.584Z InvocationID=e8d0e91e-d9da-40c1-87a5-187430b230d9 ErrorCode=400 LogTimestamp=2022-04-26T06:18:09.047327Z ResponseDesc=Created Exiting. 2022-04-26T06:18:09.048Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:18:09.065Z [qtp2085002312-13] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:09.065075Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:18:09.065Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:09.065075Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:18:09.066Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userappc HTTP/1.1 SC="500" 2022-04-26T06:18:09.077Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=33cf466f-f689-4916-8d4f-c8bceae5a49d InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:09.076500Z InvocationID=edeb8879-fe2e-4782-b1a6-4eb8a8829740 LogTimestamp=2022-04-26T06:18:09.076649Z Entering 2022-04-26T06:18:09.077Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=33cf466f-f689-4916-8d4f-c8bceae5a49d InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:09.076500Z InvocationID=edeb8879-fe2e-4782-b1a6-4eb8a8829740 LogTimestamp=2022-04-26T06:18:09.076649Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:18:09.086Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:18:09.078Z InvocationID=37a08a91-88ee-4531-902a-35587147804b PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id appc 2022-04-26T06:18:09.639Z [qtp2085002312-15] WARN o.j.d.l.c.ConsistentKeyLocker RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:18:09.078Z InvocationID=37a08a91-88ee-4531-902a-35587147804b PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Lock write succeeded but took too long: duration PT0.540643S exceeded limit PT0.5S 2022-04-26T06:18:11.553Z [qtp2085002312-15] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:09.078Z InvocationID=37a08a91-88ee-4531-902a-35587147804b ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Error returned after get Consumer Data node appc. status returned is NOT_FOUND 2022-04-26T06:18:12.551Z [qtp2085002312-15] ERROR o.o.s.b.d.cassandra.CassandraClient AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "appc,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:09.078Z InvocationID=37a08a91-88ee-4531-902a-35587147804b ErrorCode=300 Failed to save entity [ConsumerEvent(timebaseduuid=a6909660-c528-11ec-814b-87bf26af1973, timestamp1=Tue Apr 26 06:18:12 GMT 2022, requestId=33cf466f-f689-4916-8d4f-c8bceae5a49d, action=AddECOMPUserCredentials, status=201, desc=OK, modifier=Jimmy Hendrix(jh0003), ecompUser=appc,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495)], error : java.lang.IllegalArgumentException: Table or materialized view consumerevent does not exist in keyspace sdcaudit at com.datastax.driver.mapping.AnnotationParser.parseEntity(AnnotationParser.java:103) at com.datastax.driver.mapping.MappingManager.getMapper(MappingManager.java:329) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:261) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:273) at org.openecomp.sdc.be.dao.cassandra.CassandraClient.save(CassandraClient.java:156) at org.openecomp.sdc.be.dao.cassandra.AuditCassandraDao.saveRecord(AuditCassandraDao.java:77) at org.openecomp.sdc.be.auditing.impl.AuditingManager.saveEventToCassandra(AuditingManager.java:74) at org.openecomp.sdc.be.auditing.impl.AuditingManager.auditEvent(AuditingManager.java:57) at org.openecomp.sdc.be.impl.ComponentsUtils.auditConsumerCredentialsEvent(ComponentsUtils.java:1105) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:108) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:18:12.552Z [qtp2085002312-15] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "appc,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:09.078Z InvocationID=37a08a91-88ee-4531-902a-35587147804b ErrorCode=400 Failed to persist to cassandra auditing event: GENERAL_ERROR 2022-04-26T06:18:13.244Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ab29329-322d-4563-8093-f6b8fe2b76f3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:13.244Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ab29329-322d-4563-8093-f6b8fe2b76f3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@29889aac 2022-04-26T06:18:13.245Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ab29329-322d-4563-8093-f6b8fe2b76f3 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:13.245Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ab29329-322d-4563-8093-f6b8fe2b76f3 PartnerName=SDC.BE The cassandra down nodes number is 0 2022-04-26T06:18:14.040Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "appc,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=4961 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:09.078Z InvocationID=37a08a91-88ee-4531-902a-35587147804b ErrorCode=400 LogTimestamp=2022-04-26T06:18:14.039764Z ResponseDesc=Created Exiting. 2022-04-26T06:18:14.041Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:18:14.052Z [qtp2085002312-13] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:14.052237Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:18:14.052Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:14.052237Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:18:14.054Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdcae HTTP/1.1 SC="500" 2022-04-26T06:18:14.063Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c6b23769-afcb-4b97-97bb-4af84e5077c4 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:14.062985Z InvocationID=d6796f2c-d269-4dff-b5ae-b140d7796d17 LogTimestamp=2022-04-26T06:18:14.063077Z Entering 2022-04-26T06:18:14.063Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c6b23769-afcb-4b97-97bb-4af84e5077c4 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:14.062985Z InvocationID=d6796f2c-d269-4dff-b5ae-b140d7796d17 LogTimestamp=2022-04-26T06:18:14.063077Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:18:14.069Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:18:14.064Z InvocationID=c68b5872-66f9-44f8-bcb2-6f8e05dda3b9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id dcae 2022-04-26T06:18:14.845Z [qtp2085002312-12] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:14.064Z InvocationID=c68b5872-66f9-44f8-bcb2-6f8e05dda3b9 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Error returned after get Consumer Data node dcae. status returned is NOT_FOUND 2022-04-26T06:18:16.252Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=0ab29329-322d-4563-8093-f6b8fe2b76f3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:18:16.254Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=0ab29329-322d-4563-8093-f6b8fe2b76f3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:18:16.255Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=0ab29329-322d-4563-8093-f6b8fe2b76f3 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:18:16.255Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=0ab29329-322d-4563-8093-f6b8fe2b76f3 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:18:16.847Z [qtp2085002312-12] ERROR o.o.s.b.d.cassandra.CassandraClient AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "dcae,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:14.064Z InvocationID=c68b5872-66f9-44f8-bcb2-6f8e05dda3b9 ErrorCode=300 Failed to save entity [ConsumerEvent(timebaseduuid=a9201ae0-c528-11ec-814b-87bf26af1973, timestamp1=Tue Apr 26 06:18:16 GMT 2022, requestId=c6b23769-afcb-4b97-97bb-4af84e5077c4, action=AddECOMPUserCredentials, status=201, desc=OK, modifier=Jimmy Hendrix(jh0003), ecompUser=dcae,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495)], error : java.lang.IllegalArgumentException: Table or materialized view consumerevent does not exist in keyspace sdcaudit at com.datastax.driver.mapping.AnnotationParser.parseEntity(AnnotationParser.java:103) at com.datastax.driver.mapping.MappingManager.getMapper(MappingManager.java:329) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:261) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:273) at org.openecomp.sdc.be.dao.cassandra.CassandraClient.save(CassandraClient.java:156) at org.openecomp.sdc.be.dao.cassandra.AuditCassandraDao.saveRecord(AuditCassandraDao.java:77) at org.openecomp.sdc.be.auditing.impl.AuditingManager.saveEventToCassandra(AuditingManager.java:74) at org.openecomp.sdc.be.auditing.impl.AuditingManager.auditEvent(AuditingManager.java:57) at org.openecomp.sdc.be.impl.ComponentsUtils.auditConsumerCredentialsEvent(ComponentsUtils.java:1105) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:108) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:18:16.848Z [qtp2085002312-12] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "dcae,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:14.064Z InvocationID=c68b5872-66f9-44f8-bcb2-6f8e05dda3b9 ErrorCode=400 Failed to persist to cassandra auditing event: GENERAL_ERROR 2022-04-26T06:18:18.541Z [qtp2085002312-12] WARN o.j.d.l.c.ConsistentKeyLocker AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "dcae,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:14.064Z InvocationID=c68b5872-66f9-44f8-bcb2-6f8e05dda3b9 ErrorCode=400 Lock write succeeded but took too long: duration PT1.092705S exceeded limit PT0.5S 2022-04-26T06:18:20.255Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "dcae,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=6191 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:14.064Z InvocationID=c68b5872-66f9-44f8-bcb2-6f8e05dda3b9 ErrorCode=400 LogTimestamp=2022-04-26T06:18:20.255206Z ResponseDesc=Created Exiting. 2022-04-26T06:18:20.257Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:18:20.267Z [qtp2085002312-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:20.266935Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:18:20.267Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:20.266935Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:18:20.268Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermso HTTP/1.1 SC="500" 2022-04-26T06:18:20.277Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ae4b89bd-9b52-4f33-b4d6-2224ae5ed8f9 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:20.277090Z InvocationID=c9e072bd-5ba1-4f72-b01b-32544a285ab3 LogTimestamp=2022-04-26T06:18:20.277267Z Entering 2022-04-26T06:18:20.278Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ae4b89bd-9b52-4f33-b4d6-2224ae5ed8f9 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:20.277090Z InvocationID=c9e072bd-5ba1-4f72-b01b-32544a285ab3 LogTimestamp=2022-04-26T06:18:20.277267Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:18:20.284Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:18:20.278Z InvocationID=37dcc3b0-5df8-4d91-90a5-6df90008394c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id mso 2022-04-26T06:18:20.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:18:20.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:18:21.940Z [qtp2085002312-15] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:20.278Z InvocationID=37dcc3b0-5df8-4d91-90a5-6df90008394c ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Error returned after get Consumer Data node mso. status returned is NOT_FOUND 2022-04-26T06:18:23.668Z [qtp2085002312-15] ERROR o.o.s.b.d.cassandra.CassandraClient AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "mso,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:20.278Z InvocationID=37dcc3b0-5df8-4d91-90a5-6df90008394c ErrorCode=300 Failed to save entity [ConsumerEvent(timebaseduuid=ad30e830-c528-11ec-814b-87bf26af1973, timestamp1=Tue Apr 26 06:18:23 GMT 2022, requestId=ae4b89bd-9b52-4f33-b4d6-2224ae5ed8f9, action=AddECOMPUserCredentials, status=201, desc=OK, modifier=Jimmy Hendrix(jh0003), ecompUser=mso,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495)], error : java.lang.IllegalArgumentException: Table or materialized view consumerevent does not exist in keyspace sdcaudit at com.datastax.driver.mapping.AnnotationParser.parseEntity(AnnotationParser.java:103) at com.datastax.driver.mapping.MappingManager.getMapper(MappingManager.java:329) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:261) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:273) at org.openecomp.sdc.be.dao.cassandra.CassandraClient.save(CassandraClient.java:156) at org.openecomp.sdc.be.dao.cassandra.AuditCassandraDao.saveRecord(AuditCassandraDao.java:77) at org.openecomp.sdc.be.auditing.impl.AuditingManager.saveEventToCassandra(AuditingManager.java:74) at org.openecomp.sdc.be.auditing.impl.AuditingManager.auditEvent(AuditingManager.java:57) at org.openecomp.sdc.be.impl.ComponentsUtils.auditConsumerCredentialsEvent(ComponentsUtils.java:1105) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:108) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:18:23.669Z [qtp2085002312-15] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "mso,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:20.278Z InvocationID=37dcc3b0-5df8-4d91-90a5-6df90008394c ErrorCode=400 Failed to persist to cassandra auditing event: GENERAL_ERROR 2022-04-26T06:18:24.849Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3106dd07-865d-4885-97d6-5413249113e0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:24.849Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3106dd07-865d-4885-97d6-5413249113e0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4be2e572 2022-04-26T06:18:24.850Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3106dd07-865d-4885-97d6-5413249113e0 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:24.850Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3106dd07-865d-4885-97d6-5413249113e0 PartnerName=SDC.BE The cassandra down nodes number is 0 2022-04-26T06:18:26.051Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "mso,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=5773 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:20.278Z InvocationID=37dcc3b0-5df8-4d91-90a5-6df90008394c ErrorCode=400 LogTimestamp=2022-04-26T06:18:26.051367Z ResponseDesc=Created Exiting. 2022-04-26T06:18:26.052Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:18:26.062Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=8190 PartnerName=kube-probe/1.22 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=ERROR ServerIPAddress=10.233.65.41 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2022-04-26T06:18:17.872Z InvocationID=55999213-6667-4ce9-8bd3-bc6f35f44ff8 ErrorCode=500 LogTimestamp=2022-04-26T06:18:26.062143Z ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:18:26.063Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usersdnc HTTP/1.1 SC="500" 2022-04-26T06:18:26.070Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=657fea04-c07a-45e8-bb21-8697f808a313 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:26.070591Z InvocationID=5d8f83d9-196e-402b-b144-5da5bbfaca11 LogTimestamp=2022-04-26T06:18:26.070687Z Entering 2022-04-26T06:18:26.071Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=657fea04-c07a-45e8-bb21-8697f808a313 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:26.070591Z InvocationID=5d8f83d9-196e-402b-b144-5da5bbfaca11 LogTimestamp=2022-04-26T06:18:26.070687Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:18:26.076Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:18:26.071Z InvocationID=5a880302-a5bf-4ec7-98fc-6d45baf8d43e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id sdnc 2022-04-26T06:18:27.052Z [qtp2085002312-14] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:26.071Z InvocationID=5a880302-a5bf-4ec7-98fc-6d45baf8d43e ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Error returned after get Consumer Data node sdnc. status returned is NOT_FOUND 2022-04-26T06:18:27.893Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3106dd07-865d-4885-97d6-5413249113e0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:18:27.894Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3106dd07-865d-4885-97d6-5413249113e0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:18:28.250Z [qtp2085002312-14] ERROR o.o.s.b.d.cassandra.CassandraClient AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "sdnc,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:26.071Z InvocationID=5a880302-a5bf-4ec7-98fc-6d45baf8d43e ErrorCode=300 Failed to save entity [ConsumerEvent(timebaseduuid=afec1090-c528-11ec-814b-87bf26af1973, timestamp1=Tue Apr 26 06:18:28 GMT 2022, requestId=657fea04-c07a-45e8-bb21-8697f808a313, action=AddECOMPUserCredentials, status=201, desc=OK, modifier=Jimmy Hendrix(jh0003), ecompUser=sdnc,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495)], error : java.lang.IllegalArgumentException: Table or materialized view consumerevent does not exist in keyspace sdcaudit at com.datastax.driver.mapping.AnnotationParser.parseEntity(AnnotationParser.java:103) at com.datastax.driver.mapping.MappingManager.getMapper(MappingManager.java:329) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:261) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:273) at org.openecomp.sdc.be.dao.cassandra.CassandraClient.save(CassandraClient.java:156) at org.openecomp.sdc.be.dao.cassandra.AuditCassandraDao.saveRecord(AuditCassandraDao.java:77) at org.openecomp.sdc.be.auditing.impl.AuditingManager.saveEventToCassandra(AuditingManager.java:74) at org.openecomp.sdc.be.auditing.impl.AuditingManager.auditEvent(AuditingManager.java:57) at org.openecomp.sdc.be.impl.ComponentsUtils.auditConsumerCredentialsEvent(ComponentsUtils.java:1105) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:108) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:18:28.251Z [qtp2085002312-14] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "sdnc,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:26.071Z InvocationID=5a880302-a5bf-4ec7-98fc-6d45baf8d43e ErrorCode=400 Failed to persist to cassandra auditing event: GENERAL_ERROR 2022-04-26T06:18:30.550Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "sdnc,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=4479 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:26.071Z InvocationID=5a880302-a5bf-4ec7-98fc-6d45baf8d43e ErrorCode=400 LogTimestamp=2022-04-26T06:18:30.550115Z ResponseDesc=Created Exiting. 2022-04-26T06:18:30.551Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:18:30.559Z [qtp2085002312-13] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:30.559521Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:18:30.560Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:30.559521Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:18:30.560Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservid HTTP/1.1 SC="500" 2022-04-26T06:18:30.567Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=aaf42c1c-4a09-425e-9a03-c35c5e571cbe InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:30.567550Z InvocationID=dd8f90d8-ff21-41b3-b4ad-fbe7846e97a0 LogTimestamp=2022-04-26T06:18:30.567617Z Entering 2022-04-26T06:18:30.568Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=aaf42c1c-4a09-425e-9a03-c35c5e571cbe InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:30.567550Z InvocationID=dd8f90d8-ff21-41b3-b4ad-fbe7846e97a0 LogTimestamp=2022-04-26T06:18:30.567617Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:18:30.572Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:18:30.568Z InvocationID=6966972a-cacc-4062-ae76-c848a1eee64c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id vid 2022-04-26T06:18:31.946Z [qtp2085002312-16] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:30.568Z InvocationID=6966972a-cacc-4062-ae76-c848a1eee64c ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Error returned after get Consumer Data node vid. status returned is NOT_FOUND 2022-04-26T06:18:32.842Z [qtp2085002312-16] WARN o.j.d.l.c.ConsistentKeyLocker RequestID=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:30.568Z InvocationID=6966972a-cacc-4062-ae76-c848a1eee64c ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Lock write succeeded but took too long: duration PT0.888247S exceeded limit PT0.5S 2022-04-26T06:18:33.436Z [qtp2085002312-16] ERROR o.o.s.b.d.cassandra.CassandraClient AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "vid,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:30.568Z InvocationID=6966972a-cacc-4062-ae76-c848a1eee64c ErrorCode=300 Failed to save entity [ConsumerEvent(timebaseduuid=b30362b0-c528-11ec-814b-87bf26af1973, timestamp1=Tue Apr 26 06:18:33 GMT 2022, requestId=aaf42c1c-4a09-425e-9a03-c35c5e571cbe, action=AddECOMPUserCredentials, status=201, desc=OK, modifier=Jimmy Hendrix(jh0003), ecompUser=vid,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495)], error : java.lang.IllegalArgumentException: Table or materialized view consumerevent does not exist in keyspace sdcaudit at com.datastax.driver.mapping.AnnotationParser.parseEntity(AnnotationParser.java:103) at com.datastax.driver.mapping.MappingManager.getMapper(MappingManager.java:329) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:261) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:273) at org.openecomp.sdc.be.dao.cassandra.CassandraClient.save(CassandraClient.java:156) at org.openecomp.sdc.be.dao.cassandra.AuditCassandraDao.saveRecord(AuditCassandraDao.java:77) at org.openecomp.sdc.be.auditing.impl.AuditingManager.saveEventToCassandra(AuditingManager.java:74) at org.openecomp.sdc.be.auditing.impl.AuditingManager.auditEvent(AuditingManager.java:57) at org.openecomp.sdc.be.impl.ComponentsUtils.auditConsumerCredentialsEvent(ComponentsUtils.java:1105) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:108) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:18:33.437Z [qtp2085002312-16] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "vid,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:30.568Z InvocationID=6966972a-cacc-4062-ae76-c848a1eee64c ErrorCode=400 Failed to persist to cassandra auditing event: GENERAL_ERROR 2022-04-26T06:18:35.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:18:35.381Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:18:35.546Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6042a6ba-2f50-4401-8079-186ad3528880 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:35.547Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6042a6ba-2f50-4401-8079-186ad3528880 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@58de8dc3 2022-04-26T06:18:35.547Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6042a6ba-2f50-4401-8079-186ad3528880 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:35.547Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6042a6ba-2f50-4401-8079-186ad3528880 PartnerName=SDC.BE The cassandra down nodes number is 0 2022-04-26T06:18:36.359Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=6042a6ba-2f50-4401-8079-186ad3528880 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:18:36.359Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=6042a6ba-2f50-4401-8079-186ad3528880 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:18:36.440Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "vid,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=5872 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:30.568Z InvocationID=6966972a-cacc-4062-ae76-c848a1eee64c ErrorCode=400 LogTimestamp=2022-04-26T06:18:36.440455Z ResponseDesc=Created Exiting. 2022-04-26T06:18:36.441Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:18:36.450Z [qtp2085002312-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:36.450064Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:18:36.450Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:36.450064Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:18:36.450Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercognita HTTP/1.1 SC="500" 2022-04-26T06:18:36.457Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=4242c0e0-d6f3-40a7-bfac-153ca9a1f3a2 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:36.457355Z InvocationID=0d2e2274-eb97-4d86-ac20-a4546f9226c1 LogTimestamp=2022-04-26T06:18:36.457464Z Entering 2022-04-26T06:18:36.458Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=4242c0e0-d6f3-40a7-bfac-153ca9a1f3a2 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:36.457355Z InvocationID=0d2e2274-eb97-4d86-ac20-a4546f9226c1 LogTimestamp=2022-04-26T06:18:36.457464Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:18:36.462Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:18:36.458Z InvocationID=8e3020e1-6ad3-4a4e-871d-9aa0d1f2894c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id cognita 2022-04-26T06:18:37.884Z [qtp2085002312-17] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:36.458Z InvocationID=8e3020e1-6ad3-4a4e-871d-9aa0d1f2894c ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Error returned after get Consumer Data node cognita. status returned is NOT_FOUND 2022-04-26T06:18:38.462Z [qtp2085002312-17] ERROR o.o.s.b.d.cassandra.CassandraClient AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "cognita,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:36.458Z InvocationID=8e3020e1-6ad3-4a4e-871d-9aa0d1f2894c ErrorCode=300 Failed to save entity [ConsumerEvent(timebaseduuid=b6024ad0-c528-11ec-814b-87bf26af1973, timestamp1=Tue Apr 26 06:18:38 GMT 2022, requestId=4242c0e0-d6f3-40a7-bfac-153ca9a1f3a2, action=AddECOMPUserCredentials, status=201, desc=OK, modifier=Jimmy Hendrix(jh0003), ecompUser=cognita,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495)], error : java.lang.IllegalArgumentException: Table or materialized view consumerevent does not exist in keyspace sdcaudit at com.datastax.driver.mapping.AnnotationParser.parseEntity(AnnotationParser.java:103) at com.datastax.driver.mapping.MappingManager.getMapper(MappingManager.java:329) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:261) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:273) at org.openecomp.sdc.be.dao.cassandra.CassandraClient.save(CassandraClient.java:156) at org.openecomp.sdc.be.dao.cassandra.AuditCassandraDao.saveRecord(AuditCassandraDao.java:77) at org.openecomp.sdc.be.auditing.impl.AuditingManager.saveEventToCassandra(AuditingManager.java:74) at org.openecomp.sdc.be.auditing.impl.AuditingManager.auditEvent(AuditingManager.java:57) at org.openecomp.sdc.be.impl.ComponentsUtils.auditConsumerCredentialsEvent(ComponentsUtils.java:1105) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:108) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:18:38.463Z [qtp2085002312-17] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "cognita,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:36.458Z InvocationID=8e3020e1-6ad3-4a4e-871d-9aa0d1f2894c ErrorCode=400 Failed to persist to cassandra auditing event: GENERAL_ERROR 2022-04-26T06:18:38.886Z [cluster1-nio-worker-0] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.233.72.7:9042 com.datastax.driver.core.exceptions.TransportException: [cassandra.onap/10.233.72.7:9042] Connection has been closed at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:821) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:18:38.887Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.233.72.7:9042 com.datastax.driver.core.exceptions.ConnectionException: [cassandra.onap/10.233.72.7:9042] Pool was closed during initialization at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:173) at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:158) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1080) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.google.common.util.concurrent.CollectionFuture.handleAllCompleted(CollectionFuture.java:69) at com.google.common.util.concurrent.AggregateFuture.processCompleted(AggregateFuture.java:282) at com.google.common.util.concurrent.AggregateFuture.decrementCountAndMaybeComplete(AggregateFuture.java:264) at com.google.common.util.concurrent.AggregateFuture.access$200(AggregateFuture.java:42) at com.google.common.util.concurrent.AggregateFuture$1.run(AggregateFuture.java:147) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:100) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.datastax.driver.core.Connection$Future.onException(Connection.java:1518) at com.datastax.driver.core.Connection$Dispatcher.errorOutAllHandler(Connection.java:1340) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:821) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:18:38.887Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e77d0a7-401a-44a3-a5f3-f7561a43c0f1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:38.887Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e77d0a7-401a-44a3-a5f3-f7561a43c0f1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6d7802d9 2022-04-26T06:18:38.887Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e77d0a7-401a-44a3-a5f3-f7561a43c0f1 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:38.888Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e77d0a7-401a-44a3-a5f3-f7561a43c0f1 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:39.164Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "cognita,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=2706 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:36.458Z InvocationID=8e3020e1-6ad3-4a4e-871d-9aa0d1f2894c ErrorCode=400 LogTimestamp=2022-04-26T06:18:39.164142Z ResponseDesc=Created Exiting. 2022-04-26T06:18:39.165Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:18:39.174Z [qtp2085002312-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:39.174050Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:18:39.174Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:39.174050Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:18:39.175Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userclamp HTTP/1.1 SC="500" 2022-04-26T06:18:39.183Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=71dc7f17-bf6c-4174-b10f-b2a5af9c7620 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:39.183284Z InvocationID=44f9fd49-41ef-4df7-80e9-9f982380f0cc LogTimestamp=2022-04-26T06:18:39.183386Z Entering 2022-04-26T06:18:39.184Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=71dc7f17-bf6c-4174-b10f-b2a5af9c7620 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:39.183284Z InvocationID=44f9fd49-41ef-4df7-80e9-9f982380f0cc LogTimestamp=2022-04-26T06:18:39.183386Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:18:39.189Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:18:39.184Z InvocationID=e398f069-b77f-44ec-8326-6cdc2084612f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id clamp 2022-04-26T06:18:39.394Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=2e77d0a7-401a-44a3-a5f3-f7561a43c0f1 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:18:39.394Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=2e77d0a7-401a-44a3-a5f3-f7561a43c0f1 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:18:39.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f738e8b6-e8f2-4a2d-86a2-51ca3003c288 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:39.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f738e8b6-e8f2-4a2d-86a2-51ca3003c288 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@59448cb5 2022-04-26T06:18:39.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f738e8b6-e8f2-4a2d-86a2-51ca3003c288 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:39.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f738e8b6-e8f2-4a2d-86a2-51ca3003c288 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:39.738Z [qtp2085002312-14] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:39.184Z InvocationID=e398f069-b77f-44ec-8326-6cdc2084612f ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Error returned after get Consumer Data node clamp. status returned is NOT_FOUND 2022-04-26T06:18:39.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=311005f9-e26e-4d42-8069-874912904512 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:39.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=311005f9-e26e-4d42-8069-874912904512 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@75d8b2cb 2022-04-26T06:18:39.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=311005f9-e26e-4d42-8069-874912904512 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:39.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=311005f9-e26e-4d42-8069-874912904512 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:40.247Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f020b45-a467-4177-a987-43eeadc6ca81 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:40.248Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f020b45-a467-4177-a987-43eeadc6ca81 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@292e26a3 2022-04-26T06:18:40.248Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f020b45-a467-4177-a987-43eeadc6ca81 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:40.248Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f020b45-a467-4177-a987-43eeadc6ca81 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:40.270Z [qtp2085002312-14] ERROR o.o.s.b.d.cassandra.CassandraClient AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "clamp,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:39.184Z InvocationID=e398f069-b77f-44ec-8326-6cdc2084612f ErrorCode=300 Failed to save entity [ConsumerEvent(timebaseduuid=b7162bd0-c528-11ec-814b-87bf26af1973, timestamp1=Tue Apr 26 06:18:40 GMT 2022, requestId=71dc7f17-bf6c-4174-b10f-b2a5af9c7620, action=AddECOMPUserCredentials, status=201, desc=OK, modifier=Jimmy Hendrix(jh0003), ecompUser=clamp,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495)], error : java.lang.IllegalArgumentException: Table or materialized view consumerevent does not exist in keyspace sdcaudit at com.datastax.driver.mapping.AnnotationParser.parseEntity(AnnotationParser.java:103) at com.datastax.driver.mapping.MappingManager.getMapper(MappingManager.java:329) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:261) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:273) at org.openecomp.sdc.be.dao.cassandra.CassandraClient.save(CassandraClient.java:156) at org.openecomp.sdc.be.dao.cassandra.AuditCassandraDao.saveRecord(AuditCassandraDao.java:77) at org.openecomp.sdc.be.auditing.impl.AuditingManager.saveEventToCassandra(AuditingManager.java:74) at org.openecomp.sdc.be.auditing.impl.AuditingManager.auditEvent(AuditingManager.java:57) at org.openecomp.sdc.be.impl.ComponentsUtils.auditConsumerCredentialsEvent(ComponentsUtils.java:1105) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:108) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:18:40.271Z [qtp2085002312-14] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "clamp,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:39.184Z InvocationID=e398f069-b77f-44ec-8326-6cdc2084612f ErrorCode=400 Failed to persist to cassandra auditing event: GENERAL_ERROR 2022-04-26T06:18:40.462Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc2bba4e-acab-4166-bf38-e6020e8727a5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:40.462Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc2bba4e-acab-4166-bf38-e6020e8727a5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@46d9795 2022-04-26T06:18:40.462Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc2bba4e-acab-4166-bf38-e6020e8727a5 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:40.463Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc2bba4e-acab-4166-bf38-e6020e8727a5 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:40.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a04fa69-8635-4a30-afc9-104710ae1370 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:40.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a04fa69-8635-4a30-afc9-104710ae1370 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@79b18289 2022-04-26T06:18:40.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a04fa69-8635-4a30-afc9-104710ae1370 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:40.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a04fa69-8635-4a30-afc9-104710ae1370 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:40.878Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea95211d-473d-47f3-b81a-66599eef2adc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:40.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea95211d-473d-47f3-b81a-66599eef2adc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@49cf299e 2022-04-26T06:18:40.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea95211d-473d-47f3-b81a-66599eef2adc PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:40.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea95211d-473d-47f3-b81a-66599eef2adc PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:40.943Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "clamp,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=1758 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:39.184Z InvocationID=e398f069-b77f-44ec-8326-6cdc2084612f ErrorCode=400 LogTimestamp=2022-04-26T06:18:40.942406Z ResponseDesc=Created Exiting. 2022-04-26T06:18:40.943Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:18:40.954Z [qtp2085002312-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:40.954502Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:18:40.955Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:40.954502Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:18:40.957Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservfc HTTP/1.1 SC="500" 2022-04-26T06:18:40.968Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=b61b3802-e880-4f37-8a3d-dc9a3b7f02a6 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:40.968Z InvocationID=395599de-8244-4eeb-a92e-7cdb89501724 LogTimestamp=2022-04-26T06:18:40.968118Z Entering 2022-04-26T06:18:40.969Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=b61b3802-e880-4f37-8a3d-dc9a3b7f02a6 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:40.968Z InvocationID=395599de-8244-4eeb-a92e-7cdb89501724 LogTimestamp=2022-04-26T06:18:40.968118Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:18:40.979Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:18:40.970Z InvocationID=c43bc2f3-8e85-4e6b-972c-3adb17b053fe PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id vfc 2022-04-26T06:18:41.197Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36a4f95d-0a0b-4725-bcb4-fbd911b9de3f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:41.197Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36a4f95d-0a0b-4725-bcb4-fbd911b9de3f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@110c053a 2022-04-26T06:18:41.198Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36a4f95d-0a0b-4725-bcb4-fbd911b9de3f PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:41.198Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36a4f95d-0a0b-4725-bcb4-fbd911b9de3f PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:41.395Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e03d1a7-4795-4cfa-b62c-734772b1cb06 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:41.396Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e03d1a7-4795-4cfa-b62c-734772b1cb06 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7935f9b1 2022-04-26T06:18:41.396Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e03d1a7-4795-4cfa-b62c-734772b1cb06 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:41.396Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e03d1a7-4795-4cfa-b62c-734772b1cb06 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:41.549Z [qtp2085002312-16] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:40.970Z InvocationID=c43bc2f3-8e85-4e6b-972c-3adb17b053fe ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Error returned after get Consumer Data node vfc. status returned is NOT_FOUND 2022-04-26T06:18:41.611Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d6ccc104-0b46-4da1-90a2-1e6bd56786ab PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:41.611Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d6ccc104-0b46-4da1-90a2-1e6bd56786ab PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@c6599d4 2022-04-26T06:18:41.611Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d6ccc104-0b46-4da1-90a2-1e6bd56786ab PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:41.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d6ccc104-0b46-4da1-90a2-1e6bd56786ab PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:41.846Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8eba36a7-96da-4522-9873-38f6d066228d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:41.847Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8eba36a7-96da-4522-9873-38f6d066228d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3cd6d21f 2022-04-26T06:18:41.848Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8eba36a7-96da-4522-9873-38f6d066228d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:41.848Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8eba36a7-96da-4522-9873-38f6d066228d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:42.076Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4965119-c76a-49ba-b7f6-ad9a73bf8711 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:42.076Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4965119-c76a-49ba-b7f6-ad9a73bf8711 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4d02356a 2022-04-26T06:18:42.076Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4965119-c76a-49ba-b7f6-ad9a73bf8711 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:42.077Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4965119-c76a-49ba-b7f6-ad9a73bf8711 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:42.077Z [qtp2085002312-16] ERROR o.o.s.b.d.cassandra.CassandraClient AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "vfc,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:40.970Z InvocationID=c43bc2f3-8e85-4e6b-972c-3adb17b053fe ErrorCode=300 Failed to save entity [ConsumerEvent(timebaseduuid=b829e5c0-c528-11ec-814b-87bf26af1973, timestamp1=Tue Apr 26 06:18:42 GMT 2022, requestId=b61b3802-e880-4f37-8a3d-dc9a3b7f02a6, action=AddECOMPUserCredentials, status=201, desc=OK, modifier=Jimmy Hendrix(jh0003), ecompUser=vfc,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495)], error : java.lang.IllegalArgumentException: Table or materialized view consumerevent does not exist in keyspace sdcaudit at com.datastax.driver.mapping.AnnotationParser.parseEntity(AnnotationParser.java:103) at com.datastax.driver.mapping.MappingManager.getMapper(MappingManager.java:329) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:261) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:273) at org.openecomp.sdc.be.dao.cassandra.CassandraClient.save(CassandraClient.java:156) at org.openecomp.sdc.be.dao.cassandra.AuditCassandraDao.saveRecord(AuditCassandraDao.java:77) at org.openecomp.sdc.be.auditing.impl.AuditingManager.saveEventToCassandra(AuditingManager.java:74) at org.openecomp.sdc.be.auditing.impl.AuditingManager.auditEvent(AuditingManager.java:57) at org.openecomp.sdc.be.impl.ComponentsUtils.auditConsumerCredentialsEvent(ComponentsUtils.java:1105) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:108) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:18:42.077Z [qtp2085002312-16] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "vfc,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:40.970Z InvocationID=c43bc2f3-8e85-4e6b-972c-3adb17b053fe ErrorCode=400 Failed to persist to cassandra auditing event: GENERAL_ERROR 2022-04-26T06:18:42.348Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4019fbfd-7a6a-428d-9064-1a23e30661cd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:42.349Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4019fbfd-7a6a-428d-9064-1a23e30661cd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1b2cb50a 2022-04-26T06:18:42.349Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4019fbfd-7a6a-428d-9064-1a23e30661cd PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:42.349Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4019fbfd-7a6a-428d-9064-1a23e30661cd PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:42.557Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2f3449d-8785-4159-8781-fb086929826b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:42.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2f3449d-8785-4159-8781-fb086929826b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3b273482 2022-04-26T06:18:42.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2f3449d-8785-4159-8781-fb086929826b PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:42.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2f3449d-8785-4159-8781-fb086929826b PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:42.666Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "vfc,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=1696 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:40.970Z InvocationID=c43bc2f3-8e85-4e6b-972c-3adb17b053fe ErrorCode=400 LogTimestamp=2022-04-26T06:18:42.666079Z ResponseDesc=Created Exiting. 2022-04-26T06:18:42.667Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:18:42.677Z [qtp2085002312-13] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:42.677008Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:18:42.677Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:42.677008Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:18:42.678Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userworkflow HTTP/1.1 SC="500" 2022-04-26T06:18:42.685Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=10aec4e8-9d54-4450-a053-e36f4af53b44 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:42.685315Z InvocationID=409aa175-4c69-457d-8b2b-0f6526b3e6a6 LogTimestamp=2022-04-26T06:18:42.685397Z Entering 2022-04-26T06:18:42.686Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=10aec4e8-9d54-4450-a053-e36f4af53b44 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:42.685315Z InvocationID=409aa175-4c69-457d-8b2b-0f6526b3e6a6 LogTimestamp=2022-04-26T06:18:42.685397Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:18:42.690Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:18:42.686Z InvocationID=5bca0748-82b7-4e78-9029-33589a7b86f4 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id workflow 2022-04-26T06:18:42.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b808a7a-5294-4948-90e1-effd5d5be7fd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:42.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b808a7a-5294-4948-90e1-effd5d5be7fd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@316b5fd3 2022-04-26T06:18:42.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b808a7a-5294-4948-90e1-effd5d5be7fd PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:42.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b808a7a-5294-4948-90e1-effd5d5be7fd PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:43.129Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e701276b-abe2-434b-8a8d-0e41ef58a57b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:43.129Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e701276b-abe2-434b-8a8d-0e41ef58a57b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6d9e95f1 2022-04-26T06:18:43.130Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e701276b-abe2-434b-8a8d-0e41ef58a57b PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:43.130Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e701276b-abe2-434b-8a8d-0e41ef58a57b PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:43.352Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cd25270c-7f0d-422c-82fb-19d6f697ea29 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:43.352Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cd25270c-7f0d-422c-82fb-19d6f697ea29 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@13e4f9ad 2022-04-26T06:18:43.353Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cd25270c-7f0d-422c-82fb-19d6f697ea29 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:43.353Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cd25270c-7f0d-422c-82fb-19d6f697ea29 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:43.566Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb93c12b-c883-4ecf-a30c-306d70e1a79f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:43.567Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb93c12b-c883-4ecf-a30c-306d70e1a79f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@11654c77 2022-04-26T06:18:43.567Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb93c12b-c883-4ecf-a30c-306d70e1a79f PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:43.567Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb93c12b-c883-4ecf-a30c-306d70e1a79f PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:43.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6258978b-e0a6-4682-b4ca-ba285c96415c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:43.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6258978b-e0a6-4682-b4ca-ba285c96415c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7a9b0c4f 2022-04-26T06:18:43.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6258978b-e0a6-4682-b4ca-ba285c96415c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:43.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6258978b-e0a6-4682-b4ca-ba285c96415c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:43.990Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1fc02c44-a090-4e36-83d1-1d542b023dd8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:43.990Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1fc02c44-a090-4e36-83d1-1d542b023dd8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@63333ff0 2022-04-26T06:18:43.990Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1fc02c44-a090-4e36-83d1-1d542b023dd8 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:43.991Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1fc02c44-a090-4e36-83d1-1d542b023dd8 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:44.009Z [qtp2085002312-17] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:42.686Z InvocationID=5bca0748-82b7-4e78-9029-33589a7b86f4 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Error returned after get Consumer Data node workflow. status returned is NOT_FOUND 2022-04-26T06:18:44.261Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1cd7346-4fad-4608-b100-305ef18074c0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:44.261Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1cd7346-4fad-4608-b100-305ef18074c0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@57ae8baa 2022-04-26T06:18:44.262Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1cd7346-4fad-4608-b100-305ef18074c0 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:44.262Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1cd7346-4fad-4608-b100-305ef18074c0 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:44.546Z [qtp2085002312-17] ERROR o.o.s.b.d.cassandra.CassandraClient AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "workflow,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:42.686Z InvocationID=5bca0748-82b7-4e78-9029-33589a7b86f4 ErrorCode=300 Failed to save entity [ConsumerEvent(timebaseduuid=b9a2a310-c528-11ec-814b-87bf26af1973, timestamp1=Tue Apr 26 06:18:44 GMT 2022, requestId=10aec4e8-9d54-4450-a053-e36f4af53b44, action=AddECOMPUserCredentials, status=201, desc=OK, modifier=Jimmy Hendrix(jh0003), ecompUser=workflow,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495)], error : java.lang.IllegalArgumentException: Table or materialized view consumerevent does not exist in keyspace sdcaudit at com.datastax.driver.mapping.AnnotationParser.parseEntity(AnnotationParser.java:103) at com.datastax.driver.mapping.MappingManager.getMapper(MappingManager.java:329) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:261) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:273) at org.openecomp.sdc.be.dao.cassandra.CassandraClient.save(CassandraClient.java:156) at org.openecomp.sdc.be.dao.cassandra.AuditCassandraDao.saveRecord(AuditCassandraDao.java:77) at org.openecomp.sdc.be.auditing.impl.AuditingManager.saveEventToCassandra(AuditingManager.java:74) at org.openecomp.sdc.be.auditing.impl.AuditingManager.auditEvent(AuditingManager.java:57) at org.openecomp.sdc.be.impl.ComponentsUtils.auditConsumerCredentialsEvent(ComponentsUtils.java:1105) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:108) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:18:44.547Z [qtp2085002312-17] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "workflow,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:42.686Z InvocationID=5bca0748-82b7-4e78-9029-33589a7b86f4 ErrorCode=400 Failed to persist to cassandra auditing event: GENERAL_ERROR 2022-04-26T06:18:44.738Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6fe823fc-116b-4494-bc03-c2b60f3dac0c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:44.738Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6fe823fc-116b-4494-bc03-c2b60f3dac0c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6c1363dc 2022-04-26T06:18:44.738Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6fe823fc-116b-4494-bc03-c2b60f3dac0c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:44.738Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6fe823fc-116b-4494-bc03-c2b60f3dac0c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:44.905Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c6708cd-e2b6-4aef-90ee-a63fd2daab4d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:44.905Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c6708cd-e2b6-4aef-90ee-a63fd2daab4d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4ec8e365 2022-04-26T06:18:44.905Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c6708cd-e2b6-4aef-90ee-a63fd2daab4d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:44.905Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c6708cd-e2b6-4aef-90ee-a63fd2daab4d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:45.119Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e110e2cc-d3cf-4519-9664-2ac6459b5092 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:45.119Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e110e2cc-d3cf-4519-9664-2ac6459b5092 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@34fe99f3 2022-04-26T06:18:45.120Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e110e2cc-d3cf-4519-9664-2ac6459b5092 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:45.120Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e110e2cc-d3cf-4519-9664-2ac6459b5092 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:45.192Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "workflow,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=2506 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:42.686Z InvocationID=5bca0748-82b7-4e78-9029-33589a7b86f4 ErrorCode=400 LogTimestamp=2022-04-26T06:18:45.192426Z ResponseDesc=Created Exiting. 2022-04-26T06:18:45.193Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:18:45.204Z [qtp2085002312-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:45.203921Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:18:45.204Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:45.203921Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:18:45.204Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpolicy HTTP/1.1 SC="500" 2022-04-26T06:18:45.221Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a6a9b10b-7e2e-4622-943a-1b72f01987d9 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:45.220719Z InvocationID=d19b34a6-6172-417e-9dcb-500781de4f9d LogTimestamp=2022-04-26T06:18:45.220838Z Entering 2022-04-26T06:18:45.221Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a6a9b10b-7e2e-4622-943a-1b72f01987d9 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:45.220719Z InvocationID=d19b34a6-6172-417e-9dcb-500781de4f9d LogTimestamp=2022-04-26T06:18:45.220838Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:18:45.228Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:18:45.222Z InvocationID=3b0077ec-d027-44e5-beff-a6fa13dd7c60 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id policy 2022-04-26T06:18:45.388Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46475a75-8860-48a5-aae0-4adee2264ba8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:45.388Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46475a75-8860-48a5-aae0-4adee2264ba8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@17aea71d 2022-04-26T06:18:45.389Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46475a75-8860-48a5-aae0-4adee2264ba8 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:45.389Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46475a75-8860-48a5-aae0-4adee2264ba8 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:45.600Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10e913ec-9b82-40d9-b8d8-88e564695e1a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:45.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10e913ec-9b82-40d9-b8d8-88e564695e1a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@60542bb7 2022-04-26T06:18:45.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10e913ec-9b82-40d9-b8d8-88e564695e1a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:45.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10e913ec-9b82-40d9-b8d8-88e564695e1a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:45.769Z [qtp2085002312-13] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:45.222Z InvocationID=3b0077ec-d027-44e5-beff-a6fa13dd7c60 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Error returned after get Consumer Data node policy. status returned is NOT_FOUND 2022-04-26T06:18:45.996Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7541c187-c649-42ba-85b8-996a2f7222e6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:45.996Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7541c187-c649-42ba-85b8-996a2f7222e6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@698e2f13 2022-04-26T06:18:45.996Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7541c187-c649-42ba-85b8-996a2f7222e6 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:45.996Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7541c187-c649-42ba-85b8-996a2f7222e6 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:46.201Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae51703e-f53f-4ab1-a9b9-4c37685dad85 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:46.202Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae51703e-f53f-4ab1-a9b9-4c37685dad85 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@29ff02d0 2022-04-26T06:18:46.202Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae51703e-f53f-4ab1-a9b9-4c37685dad85 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:46.202Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae51703e-f53f-4ab1-a9b9-4c37685dad85 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:46.361Z [qtp2085002312-13] ERROR o.o.s.b.d.cassandra.CassandraClient AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "policy,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:45.222Z InvocationID=3b0077ec-d027-44e5-beff-a6fa13dd7c60 ErrorCode=300 Failed to save entity [ConsumerEvent(timebaseduuid=bab79580-c528-11ec-814b-87bf26af1973, timestamp1=Tue Apr 26 06:18:46 GMT 2022, requestId=a6a9b10b-7e2e-4622-943a-1b72f01987d9, action=AddECOMPUserCredentials, status=201, desc=OK, modifier=Jimmy Hendrix(jh0003), ecompUser=policy,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495)], error : java.lang.IllegalArgumentException: Table or materialized view consumerevent does not exist in keyspace sdcaudit at com.datastax.driver.mapping.AnnotationParser.parseEntity(AnnotationParser.java:103) at com.datastax.driver.mapping.MappingManager.getMapper(MappingManager.java:329) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:261) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:273) at org.openecomp.sdc.be.dao.cassandra.CassandraClient.save(CassandraClient.java:156) at org.openecomp.sdc.be.dao.cassandra.AuditCassandraDao.saveRecord(AuditCassandraDao.java:77) at org.openecomp.sdc.be.auditing.impl.AuditingManager.saveEventToCassandra(AuditingManager.java:74) at org.openecomp.sdc.be.auditing.impl.AuditingManager.auditEvent(AuditingManager.java:57) at org.openecomp.sdc.be.impl.ComponentsUtils.auditConsumerCredentialsEvent(ComponentsUtils.java:1105) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:108) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:18:46.362Z [qtp2085002312-13] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "policy,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:45.222Z InvocationID=3b0077ec-d027-44e5-beff-a6fa13dd7c60 ErrorCode=400 Failed to persist to cassandra auditing event: GENERAL_ERROR 2022-04-26T06:18:46.471Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=99a5a12d-99d6-47a6-9df7-1fead135748c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:46.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=99a5a12d-99d6-47a6-9df7-1fead135748c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4b0bcebd 2022-04-26T06:18:46.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=99a5a12d-99d6-47a6-9df7-1fead135748c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:46.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=99a5a12d-99d6-47a6-9df7-1fead135748c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:46.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dcfd23a-14ca-4203-8e90-e72cf4888496 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:46.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dcfd23a-14ca-4203-8e90-e72cf4888496 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@a434df8 2022-04-26T06:18:46.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dcfd23a-14ca-4203-8e90-e72cf4888496 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:46.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dcfd23a-14ca-4203-8e90-e72cf4888496 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:46.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fafddbd7-2dee-4ae7-99fd-666ed678fba3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:46.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fafddbd7-2dee-4ae7-99fd-666ed678fba3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@122d1d14 2022-04-26T06:18:46.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fafddbd7-2dee-4ae7-99fd-666ed678fba3 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:46.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fafddbd7-2dee-4ae7-99fd-666ed678fba3 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:47.052Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "policy,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=1830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:45.222Z InvocationID=3b0077ec-d027-44e5-beff-a6fa13dd7c60 ErrorCode=400 LogTimestamp=2022-04-26T06:18:47.052324Z ResponseDesc=Created Exiting. 2022-04-26T06:18:47.053Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:18:47.063Z [qtp2085002312-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:47.063491Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:18:47.063Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:47.063491Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:18:47.064Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpomba HTTP/1.1 SC="500" 2022-04-26T06:18:47.072Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5e7cf0d4-c988-4456-a4c4-f659d6620173 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:47.071633Z InvocationID=1a58551c-4754-4df2-97e6-60d7ddda7edb LogTimestamp=2022-04-26T06:18:47.071769Z Entering 2022-04-26T06:18:47.072Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5e7cf0d4-c988-4456-a4c4-f659d6620173 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:47.071633Z InvocationID=1a58551c-4754-4df2-97e6-60d7ddda7edb LogTimestamp=2022-04-26T06:18:47.071769Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:18:47.077Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:18:47.072Z InvocationID=9ce162e8-4044-46fc-a98e-d25018b03aaf PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id pomba 2022-04-26T06:18:47.093Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b3471d7-52d7-4021-b83c-5159e1f419fd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:47.094Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b3471d7-52d7-4021-b83c-5159e1f419fd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@43957403 2022-04-26T06:18:47.094Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b3471d7-52d7-4021-b83c-5159e1f419fd PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:47.094Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b3471d7-52d7-4021-b83c-5159e1f419fd PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:47.303Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a1d9ada-1cb4-4be9-b06d-2330a488fd5e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:47.304Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a1d9ada-1cb4-4be9-b06d-2330a488fd5e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@567e24ea 2022-04-26T06:18:47.304Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a1d9ada-1cb4-4be9-b06d-2330a488fd5e PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:47.305Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a1d9ada-1cb4-4be9-b06d-2330a488fd5e PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:47.497Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=13684504-cffb-439f-a610-dbb394fceddb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:47.498Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=13684504-cffb-439f-a610-dbb394fceddb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@46c29f50 2022-04-26T06:18:47.498Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=13684504-cffb-439f-a610-dbb394fceddb PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:47.498Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=13684504-cffb-439f-a610-dbb394fceddb PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:47.646Z [qtp2085002312-18] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:47.072Z InvocationID=9ce162e8-4044-46fc-a98e-d25018b03aaf ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Error returned after get Consumer Data node pomba. status returned is NOT_FOUND 2022-04-26T06:18:47.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=daa27977-a890-4703-9064-cb6a1213c051 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:47.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=daa27977-a890-4703-9064-cb6a1213c051 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3c7def21 2022-04-26T06:18:47.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=daa27977-a890-4703-9064-cb6a1213c051 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:47.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=daa27977-a890-4703-9064-cb6a1213c051 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:47.927Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=935646ea-34c1-409b-8284-c89eae4c13fe PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:47.927Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=935646ea-34c1-409b-8284-c89eae4c13fe PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@54a0b1c1 2022-04-26T06:18:47.928Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=935646ea-34c1-409b-8284-c89eae4c13fe PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:47.928Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=935646ea-34c1-409b-8284-c89eae4c13fe PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:48.153Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4960f5b1-0a03-4ff6-a817-d1b2ff12fe1a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:48.154Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4960f5b1-0a03-4ff6-a817-d1b2ff12fe1a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@73de86cb 2022-04-26T06:18:48.154Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4960f5b1-0a03-4ff6-a817-d1b2ff12fe1a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:48.154Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4960f5b1-0a03-4ff6-a817-d1b2ff12fe1a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:48.171Z [qtp2085002312-18] ERROR o.o.s.b.d.cassandra.CassandraClient AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "pomba,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:47.072Z InvocationID=9ce162e8-4044-46fc-a98e-d25018b03aaf ErrorCode=300 Failed to save entity [ConsumerEvent(timebaseduuid=bbcbc4a0-c528-11ec-814b-87bf26af1973, timestamp1=Tue Apr 26 06:18:48 GMT 2022, requestId=5e7cf0d4-c988-4456-a4c4-f659d6620173, action=AddECOMPUserCredentials, status=201, desc=OK, modifier=Jimmy Hendrix(jh0003), ecompUser=pomba,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495)], error : java.lang.IllegalArgumentException: Table or materialized view consumerevent does not exist in keyspace sdcaudit at com.datastax.driver.mapping.AnnotationParser.parseEntity(AnnotationParser.java:103) at com.datastax.driver.mapping.MappingManager.getMapper(MappingManager.java:329) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:261) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:273) at org.openecomp.sdc.be.dao.cassandra.CassandraClient.save(CassandraClient.java:156) at org.openecomp.sdc.be.dao.cassandra.AuditCassandraDao.saveRecord(AuditCassandraDao.java:77) at org.openecomp.sdc.be.auditing.impl.AuditingManager.saveEventToCassandra(AuditingManager.java:74) at org.openecomp.sdc.be.auditing.impl.AuditingManager.auditEvent(AuditingManager.java:57) at org.openecomp.sdc.be.impl.ComponentsUtils.auditConsumerCredentialsEvent(ComponentsUtils.java:1105) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:108) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:18:48.172Z [qtp2085002312-18] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "pomba,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:47.072Z InvocationID=9ce162e8-4044-46fc-a98e-d25018b03aaf ErrorCode=400 Failed to persist to cassandra auditing event: GENERAL_ERROR 2022-04-26T06:18:48.491Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a52fda72-1d4e-4c67-b47b-8128d9313506 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:48.491Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a52fda72-1d4e-4c67-b47b-8128d9313506 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@21b4bfff 2022-04-26T06:18:48.492Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a52fda72-1d4e-4c67-b47b-8128d9313506 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:48.492Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a52fda72-1d4e-4c67-b47b-8128d9313506 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:48.699Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8357316e-7350-404c-b2a5-1095d8c46bf2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:48.700Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8357316e-7350-404c-b2a5-1095d8c46bf2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7263480f 2022-04-26T06:18:48.700Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8357316e-7350-404c-b2a5-1095d8c46bf2 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:48.700Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8357316e-7350-404c-b2a5-1095d8c46bf2 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:48.854Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "pomba,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=1782 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:47.072Z InvocationID=9ce162e8-4044-46fc-a98e-d25018b03aaf ErrorCode=400 LogTimestamp=2022-04-26T06:18:48.854114Z ResponseDesc=Created Exiting. 2022-04-26T06:18:48.855Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:18:48.863Z [qtp2085002312-14] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:48.862919Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:18:48.863Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:48.862919Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:18:48.863Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermulticloud HTTP/1.1 SC="500" 2022-04-26T06:18:48.871Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=be3865fc-985c-4a85-9337-bb10fb99483f InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:48.871013Z InvocationID=c7b93fcd-a767-4b9f-8398-e57b193919dd LogTimestamp=2022-04-26T06:18:48.871106Z Entering 2022-04-26T06:18:48.871Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=be3865fc-985c-4a85-9337-bb10fb99483f InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:48.871013Z InvocationID=c7b93fcd-a767-4b9f-8398-e57b193919dd LogTimestamp=2022-04-26T06:18:48.871106Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:18:48.876Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:18:48.872Z InvocationID=904aa063-b4b1-4029-b75e-1df144fc7a80 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id multicloud 2022-04-26T06:18:48.956Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=897421a4-4da9-453b-8e54-d6d7f82a75ab PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:48.956Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=897421a4-4da9-453b-8e54-d6d7f82a75ab PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@239cd13b 2022-04-26T06:18:48.956Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=897421a4-4da9-453b-8e54-d6d7f82a75ab PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:48.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=897421a4-4da9-453b-8e54-d6d7f82a75ab PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:49.172Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96f6faa1-9014-459c-a6b9-4678553b22e6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:49.172Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96f6faa1-9014-459c-a6b9-4678553b22e6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2473f35d 2022-04-26T06:18:49.173Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96f6faa1-9014-459c-a6b9-4678553b22e6 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:49.173Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96f6faa1-9014-459c-a6b9-4678553b22e6 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:49.528Z [qtp2085002312-16] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:48.872Z InvocationID=904aa063-b4b1-4029-b75e-1df144fc7a80 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Error returned after get Consumer Data node multicloud. status returned is NOT_FOUND 2022-04-26T06:18:49.567Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c619513-c11c-4652-b173-68cbff1d2963 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:49.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c619513-c11c-4652-b173-68cbff1d2963 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@203ceb6b 2022-04-26T06:18:49.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c619513-c11c-4652-b173-68cbff1d2963 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:49.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c619513-c11c-4652-b173-68cbff1d2963 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:50.062Z [qtp2085002312-16] ERROR o.o.s.b.d.cassandra.CassandraClient AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "multicloud,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:48.872Z InvocationID=904aa063-b4b1-4029-b75e-1df144fc7a80 ErrorCode=300 Failed to save entity [ConsumerEvent(timebaseduuid=bcec4fd0-c528-11ec-814b-87bf26af1973, timestamp1=Tue Apr 26 06:18:50 GMT 2022, requestId=be3865fc-985c-4a85-9337-bb10fb99483f, action=AddECOMPUserCredentials, status=201, desc=OK, modifier=Jimmy Hendrix(jh0003), ecompUser=multicloud,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495)], error : java.lang.IllegalArgumentException: Table or materialized view consumerevent does not exist in keyspace sdcaudit at com.datastax.driver.mapping.AnnotationParser.parseEntity(AnnotationParser.java:103) at com.datastax.driver.mapping.MappingManager.getMapper(MappingManager.java:329) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:261) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:273) at org.openecomp.sdc.be.dao.cassandra.CassandraClient.save(CassandraClient.java:156) at org.openecomp.sdc.be.dao.cassandra.AuditCassandraDao.saveRecord(AuditCassandraDao.java:77) at org.openecomp.sdc.be.auditing.impl.AuditingManager.saveEventToCassandra(AuditingManager.java:74) at org.openecomp.sdc.be.auditing.impl.AuditingManager.auditEvent(AuditingManager.java:57) at org.openecomp.sdc.be.impl.ComponentsUtils.auditConsumerCredentialsEvent(ComponentsUtils.java:1105) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:108) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:18:50.063Z [qtp2085002312-16] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "multicloud,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:48.872Z InvocationID=904aa063-b4b1-4029-b75e-1df144fc7a80 ErrorCode=400 Failed to persist to cassandra auditing event: GENERAL_ERROR 2022-04-26T06:18:50.262Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7759104-ec2d-4834-9703-d507ce3fec52 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:50.262Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7759104-ec2d-4834-9703-d507ce3fec52 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4877ec53 2022-04-26T06:18:50.262Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7759104-ec2d-4834-9703-d507ce3fec52 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:50.263Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7759104-ec2d-4834-9703-d507ce3fec52 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:50.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:18:50.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:18:50.955Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "multicloud,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=2083 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:48.872Z InvocationID=904aa063-b4b1-4029-b75e-1df144fc7a80 ErrorCode=400 LogTimestamp=2022-04-26T06:18:50.955001Z ResponseDesc=Created Exiting. 2022-04-26T06:18:50.956Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:18:50.963Z [qtp2085002312-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:50.963162Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:18:50.963Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:50.963162Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:18:50.963Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercds HTTP/1.1 SC="500" 2022-04-26T06:18:50.970Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=144f6c97-0b3b-47e1-8929-ec90b2b174b1 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:50.969933Z InvocationID=08a82a4d-93f1-44ca-8b27-69622bdc5f12 LogTimestamp=2022-04-26T06:18:50.969995Z Entering 2022-04-26T06:18:50.970Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=144f6c97-0b3b-47e1-8929-ec90b2b174b1 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:50.969933Z InvocationID=08a82a4d-93f1-44ca-8b27-69622bdc5f12 LogTimestamp=2022-04-26T06:18:50.969995Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:18:50.975Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:18:50.970Z InvocationID=f6f43d29-2c50-441f-8bc1-3fd70b73f99f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id cds 2022-04-26T06:18:51.248Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a18278b8-8206-4163-92a8-bcaf3b1eef38 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:51.248Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a18278b8-8206-4163-92a8-bcaf3b1eef38 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@d27e96f 2022-04-26T06:18:51.248Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a18278b8-8206-4163-92a8-bcaf3b1eef38 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:51.248Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a18278b8-8206-4163-92a8-bcaf3b1eef38 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:51.756Z [qtp2085002312-12] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:50.970Z InvocationID=f6f43d29-2c50-441f-8bc1-3fd70b73f99f ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Error returned after get Consumer Data node cds. status returned is NOT_FOUND 2022-04-26T06:18:52.255Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4c75431-df64-4a41-a574-619db538da28 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:52.255Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4c75431-df64-4a41-a574-619db538da28 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@36f50a8d 2022-04-26T06:18:52.255Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4c75431-df64-4a41-a574-619db538da28 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:52.256Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4c75431-df64-4a41-a574-619db538da28 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:52.360Z [qtp2085002312-12] ERROR o.o.s.b.d.cassandra.CassandraClient AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "cds,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:50.970Z InvocationID=f6f43d29-2c50-441f-8bc1-3fd70b73f99f ErrorCode=300 Failed to save entity [ConsumerEvent(timebaseduuid=be4af570-c528-11ec-814b-87bf26af1973, timestamp1=Tue Apr 26 06:18:52 GMT 2022, requestId=144f6c97-0b3b-47e1-8929-ec90b2b174b1, action=AddECOMPUserCredentials, status=201, desc=OK, modifier=Jimmy Hendrix(jh0003), ecompUser=cds,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495)], error : java.lang.IllegalArgumentException: Table or materialized view consumerevent does not exist in keyspace sdcaudit at com.datastax.driver.mapping.AnnotationParser.parseEntity(AnnotationParser.java:103) at com.datastax.driver.mapping.MappingManager.getMapper(MappingManager.java:329) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:261) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:273) at org.openecomp.sdc.be.dao.cassandra.CassandraClient.save(CassandraClient.java:156) at org.openecomp.sdc.be.dao.cassandra.AuditCassandraDao.saveRecord(AuditCassandraDao.java:77) at org.openecomp.sdc.be.auditing.impl.AuditingManager.saveEventToCassandra(AuditingManager.java:74) at org.openecomp.sdc.be.auditing.impl.AuditingManager.auditEvent(AuditingManager.java:57) at org.openecomp.sdc.be.impl.ComponentsUtils.auditConsumerCredentialsEvent(ComponentsUtils.java:1105) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:108) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:18:52.361Z [qtp2085002312-12] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "cds,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:50.970Z InvocationID=f6f43d29-2c50-441f-8bc1-3fd70b73f99f ErrorCode=400 Failed to persist to cassandra auditing event: GENERAL_ERROR 2022-04-26T06:18:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:18:52.686Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:18:52.686Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:18:53.450Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1219e07d-001b-435d-b27c-0a17ad3b0851 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:53.451Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1219e07d-001b-435d-b27c-0a17ad3b0851 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@71d5c88b 2022-04-26T06:18:53.451Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1219e07d-001b-435d-b27c-0a17ad3b0851 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:53.451Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1219e07d-001b-435d-b27c-0a17ad3b0851 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:53.458Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "cds,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=2488 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:50.970Z InvocationID=f6f43d29-2c50-441f-8bc1-3fd70b73f99f ErrorCode=400 LogTimestamp=2022-04-26T06:18:53.458626Z ResponseDesc=Created Exiting. 2022-04-26T06:18:53.459Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:18:53.469Z [qtp2085002312-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:53.469554Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:18:53.469Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:18:53.469554Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:18:53.470Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermodeling HTTP/1.1 SC="500" 2022-04-26T06:18:53.478Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=b1488f8f-89ac-4d72-8352-56ab1318cfcc InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:53.477943Z InvocationID=89921e12-b315-496b-8c57-964e91a8e90e LogTimestamp=2022-04-26T06:18:53.478033Z Entering 2022-04-26T06:18:53.478Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=b1488f8f-89ac-4d72-8352-56ab1318cfcc InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:53.477943Z InvocationID=89921e12-b315-496b-8c57-964e91a8e90e LogTimestamp=2022-04-26T06:18:53.478033Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:18:53.486Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:18:53.479Z InvocationID=343e0257-7f1f-46aa-8f92-9a033f0e3bd7 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id modeling 2022-04-26T06:18:54.247Z [qtp2085002312-18] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:53.479Z InvocationID=343e0257-7f1f-46aa-8f92-9a033f0e3bd7 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Error returned after get Consumer Data node modeling. status returned is NOT_FOUND 2022-04-26T06:18:55.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c53a134f-6ef4-40e9-b94b-12c597d64427 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:55.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c53a134f-6ef4-40e9-b94b-12c597d64427 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6b7b796 2022-04-26T06:18:55.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c53a134f-6ef4-40e9-b94b-12c597d64427 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:55.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c53a134f-6ef4-40e9-b94b-12c597d64427 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:55.848Z [qtp2085002312-18] ERROR o.o.s.b.d.cassandra.CassandraClient AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "modeling,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:53.479Z InvocationID=343e0257-7f1f-46aa-8f92-9a033f0e3bd7 ErrorCode=300 Failed to save entity [ConsumerEvent(timebaseduuid=c05f2f70-c528-11ec-814b-87bf26af1973, timestamp1=Tue Apr 26 06:18:55 GMT 2022, requestId=b1488f8f-89ac-4d72-8352-56ab1318cfcc, action=AddECOMPUserCredentials, status=201, desc=OK, modifier=Jimmy Hendrix(jh0003), ecompUser=modeling,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495)], error : java.lang.IllegalArgumentException: Table or materialized view consumerevent does not exist in keyspace sdcaudit at com.datastax.driver.mapping.AnnotationParser.parseEntity(AnnotationParser.java:103) at com.datastax.driver.mapping.MappingManager.getMapper(MappingManager.java:329) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:261) at com.datastax.driver.mapping.MappingManager.mapper(MappingManager.java:273) at org.openecomp.sdc.be.dao.cassandra.CassandraClient.save(CassandraClient.java:156) at org.openecomp.sdc.be.dao.cassandra.AuditCassandraDao.saveRecord(AuditCassandraDao.java:77) at org.openecomp.sdc.be.auditing.impl.AuditingManager.saveEventToCassandra(AuditingManager.java:74) at org.openecomp.sdc.be.auditing.impl.AuditingManager.auditEvent(AuditingManager.java:57) at org.openecomp.sdc.be.impl.ComponentsUtils.auditConsumerCredentialsEvent(ComponentsUtils.java:1105) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:108) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:18:55.849Z [qtp2085002312-18] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "modeling,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:53.479Z InvocationID=343e0257-7f1f-46aa-8f92-9a033f0e3bd7 ErrorCode=400 Failed to persist to cassandra auditing event: GENERAL_ERROR 2022-04-26T06:18:56.658Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "modeling,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=3179 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=catalog-be ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2022-04-26T06:18:53.479Z InvocationID=343e0257-7f1f-46aa-8f92-9a033f0e3bd7 ErrorCode=400 LogTimestamp=2022-04-26T06:18:56.658116Z ResponseDesc=Created Exiting. 2022-04-26T06:18:56.659Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:18:56.857Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=e81631cf-6653-4698-8e87-6f7ba76373c8 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/user/jh0003 ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:56.856759Z InvocationID=b98ed718-853d-44da-ba05-d600f295b9ec LogTimestamp=2022-04-26T06:18:56.856925Z Entering 2022-04-26T06:18:56.858Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=e81631cf-6653-4698-8e87-6f7ba76373c8 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/user/jh0003 ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:56.856759Z InvocationID=b98ed718-853d-44da-ba05-d600f295b9ec LogTimestamp=2022-04-26T06:18:56.856925Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2022-04-26T06:18:56.863Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/user/jh0003 EntryTimestamp=2022-04-26T06:18:56.858Z InvocationID=507d57a9-a2b6-4298-8561-597aaa8862bd LogTimestamp=2022-04-26T06:18:56.863152Z ResponseDesc=OK Exiting. 2022-04-26T06:18:56.863Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2022-04-26T06:18:57.278Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1943cbb0-0b7e-4515-9c4e-dd242d5c90e4 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/screen ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:57.277652Z InvocationID=5161ec07-af44-4a57-83ae-af92ec2a5e5c LogTimestamp=2022-04-26T06:18:57.277759Z Entering 2022-04-26T06:18:57.278Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1943cbb0-0b7e-4515-9c4e-dd242d5c90e4 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/screen ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:57.277652Z InvocationID=5161ec07-af44-4a57-83ae-af92ec2a5e5c LogTimestamp=2022-04-26T06:18:57.277759Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 GET /sdc2/rest/v1/screen HTTP/1.1 2022-04-26T06:18:57.460Z [qtp2085002312-15] INFO o.o.s.b.m.j.o.ToscaElementOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/screen EntryTimestamp=2022-04-26T06:18:57.279Z InvocationID=8a4250ff-8b37-4c86-8222-e50e3e8f5bb7 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb time to fetch all catalog elements: StopWatch '': running time = 74263093 ns\n---------------------------------------------\nns % Task name\n---------------------------------------------\n074263093 100% \n 2022-04-26T06:18:57.472Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=193 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/screen EntryTimestamp=2022-04-26T06:18:57.279Z InvocationID=8a4250ff-8b37-4c86-8222-e50e3e8f5bb7 LogTimestamp=2022-04-26T06:18:57.472448Z ResponseDesc=OK Exiting. 2022-04-26T06:18:57.473Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/screen HTTP/1.1 SC="200" 2022-04-26T06:18:57.483Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=b6c9b6db-459f-4458-970f-1588bd35698e InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/lock ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:57.483029Z InvocationID=eec659f5-d857-4591-8f61-0b5abf0178f2 LogTimestamp=2022-04-26T06:18:57.483124Z Entering 2022-04-26T06:18:57.483Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=b6c9b6db-459f-4458-970f-1588bd35698e InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/lock ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:57.483029Z InvocationID=eec659f5-d857-4591-8f61-0b5abf0178f2 LogTimestamp=2022-04-26T06:18:57.483124Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/lock HTTP/1.1 2022-04-26T06:18:57.529Z [qtp2085002312-13] INFO o.o.sdc.be.servlets.LockServlet RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/lock EntryTimestamp=2022-04-26T06:18:57.484Z InvocationID=08f4ce34-ee1d-4316-bdc9-4a82f392840e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb User jh0003 attempting to set disable locking with value true 2022-04-26T06:18:57.530Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/lock EntryTimestamp=2022-04-26T06:18:57.484Z InvocationID=08f4ce34-ee1d-4316-bdc9-4a82f392840e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Toggling disable locking from false to true 2022-04-26T06:18:57.531Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=46 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/lock EntryTimestamp=2022-04-26T06:18:57.484Z InvocationID=08f4ce34-ee1d-4316-bdc9-4a82f392840e LogTimestamp=2022-04-26T06:18:57.530934Z ResponseDesc=OK Exiting. 2022-04-26T06:18:57.531Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/lock HTTP/1.1 SC="200" 2022-04-26T06:18:57.548Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=9034adb8-9af2-419d-9f31-bbbc58bbd5d3 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:57.548429Z InvocationID=ecfa47eb-2ac6-4c06-b037-2def6dbe00f4 LogTimestamp=2022-04-26T06:18:57.548555Z Entering 2022-04-26T06:18:57.549Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=9034adb8-9af2-419d-9f31-bbbc58bbd5d3 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:18:57.548429Z InvocationID=ecfa47eb-2ac6-4c06-b037-2def6dbe00f4 LogTimestamp=2022-04-26T06:18:57.548555Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2022-04-26T06:18:58.145Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.Root.datatype to dao for create 2022-04-26T06:18:58.347Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c3ad775-2bdc-4040-bf8c-9cf25e763d2d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:18:58.347Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c3ad775-2bdc-4040-bf8c-9cf25e763d2d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@34695abe 2022-04-26T06:18:58.347Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c3ad775-2bdc-4040-bf8c-9cf25e763d2d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:18:58.347Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c3ad775-2bdc-4040-bf8c-9cf25e763d2d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:18:58.746Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : integer.datatype to dao for create 2022-04-26T06:19:00.131Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : string.datatype to dao for create 2022-04-26T06:19:00.648Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : boolean.datatype to dao for create 2022-04-26T06:19:01.274Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : float.datatype to dao for create 2022-04-26T06:19:01.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55a48bbb-3679-4185-8f0a-22aa9bc295a8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:19:01.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55a48bbb-3679-4185-8f0a-22aa9bc295a8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2ad69fdc 2022-04-26T06:19:01.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55a48bbb-3679-4185-8f0a-22aa9bc295a8 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:19:01.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55a48bbb-3679-4185-8f0a-22aa9bc295a8 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:19:01.735Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : list.datatype to dao for create 2022-04-26T06:19:02.329Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : map.datatype to dao for create 2022-04-26T06:19:02.848Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : json.datatype to dao for create 2022-04-26T06:19:03.239Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : scalar-unit.datatype to dao for create 2022-04-26T06:19:03.549Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : scalar-unit.size.datatype to dao for create 2022-04-26T06:19:04.035Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : scalar-unit.time.datatype to dao for create 2022-04-26T06:19:04.431Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : scalar-unit.frequency.datatype to dao for create 2022-04-26T06:19:04.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5187407-b43d-4d30-84d3-4706143a8d3b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:19:04.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5187407-b43d-4d30-84d3-4706143a8d3b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@75189b9d 2022-04-26T06:19:04.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5187407-b43d-4d30-84d3-4706143a8d3b PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:19:04.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5187407-b43d-4d30-84d3-4706143a8d3b PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:19:04.644Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.Credential.datatype to dao for create 2022-04-26T06:19:05.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:19:05.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:19:06.137Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.PortMirroringConnectionPointDescription.datatype to dao for create 2022-04-26T06:19:06.431Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.TimeInterval.datatype to dao for create 2022-04-26T06:19:06.541Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.network.NetworkInfo.datatype to dao for create 2022-04-26T06:19:06.848Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.network.PortInfo.datatype to dao for create 2022-04-26T06:19:07.146Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.network.PortDef.datatype to dao for create 2022-04-26T06:19:07.332Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.network.PortSpec.datatype to dao for create 2022-04-26T06:19:07.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a48774f1-be97-4a87-96ba-e35e1868c4bb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:19:07.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a48774f1-be97-4a87-96ba-e35e1868c4bb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4e2899af 2022-04-26T06:19:07.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a48774f1-be97-4a87-96ba-e35e1868c4bb PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:19:07.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a48774f1-be97-4a87-96ba-e35e1868c4bb PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:19:07.657Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:19:07.937Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.network.AddressPair.datatype to dao for create 2022-04-26T06:19:08.132Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.network.subnet.HostRoute.datatype to dao for create 2022-04-26T06:19:08.236Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.network.AllocationPool.datatype to dao for create 2022-04-26T06:19:08.334Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.network.neutron.Subnet.datatype to dao for create 2022-04-26T06:19:08.757Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.novaServer.network.PortExtraProperties.datatype to dao for create 2022-04-26T06:19:08.936Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.novaServer.network.AddressInfo.datatype to dao for create 2022-04-26T06:19:09.030Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.neutron.port.FixedIps.datatype to dao for create 2022-04-26T06:19:09.061Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.FileInfo.datatype to dao for create 2022-04-26T06:19:09.088Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrail.network.rule.PortPairs.datatype to dao for create 2022-04-26T06:19:09.112Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrail.network.rule.Rule.datatype to dao for create 2022-04-26T06:19:09.199Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrail.network.rule.RuleList.datatype to dao for create 2022-04-26T06:19:09.223Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrail.network.rule.VirtualNetwork.datatype to dao for create 2022-04-26T06:19:09.245Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.network.neutron.SecurityRules.Rule.datatype to dao for create 2022-04-26T06:19:09.350Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.substitution.SubstitutionFiltering.datatype to dao for create 2022-04-26T06:19:09.407Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.network.rule.RefDataSequence.datatype to dao for create 2022-04-26T06:19:09.436Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.network.rule.RefData.datatype to dao for create 2022-04-26T06:19:09.469Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.network.rule.ref.data.IpamSubnet.datatype to dao for create 2022-04-26T06:19:09.489Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.network.rule.ref.data.IpamSubnetList.datatype to dao for create 2022-04-26T06:19:09.526Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.network.rule.IpamRefData.datatype to dao for create 2022-04-26T06:19:09.545Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.network.rule.SrcVirtualNetwork.datatype to dao for create 2022-04-26T06:19:09.567Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.network.rule.DstVirtualNetwork.datatype to dao for create 2022-04-26T06:19:09.586Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.network.rule.DstPortPairs.datatype to dao for create 2022-04-26T06:19:09.613Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.network.rule.SrcPortPairs.datatype to dao for create 2022-04-26T06:19:09.637Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.network.rule.ActionList.datatype to dao for create 2022-04-26T06:19:09.669Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.network.rule.Rule.datatype to dao for create 2022-04-26T06:19:09.779Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.network.rule.RuleList.datatype to dao for create 2022-04-26T06:19:09.868Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.network.contrail.port.StaticRoute.datatype to dao for create 2022-04-26T06:19:09.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eff9d56d-a279-41b2-803a-ec3e02d7302c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:19:09.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eff9d56d-a279-41b2-803a-ec3e02d7302c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@30ea867e 2022-04-26T06:19:09.880Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eff9d56d-a279-41b2-803a-ec3e02d7302c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:19:09.880Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eff9d56d-a279-41b2-803a-ec3e02d7302c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:19:09.899Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.network.contrail.AddressPair.datatype to dao for create 2022-04-26T06:19:09.955Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.network.contrail.InterfaceData.datatype to dao for create 2022-04-26T06:19:09.992Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.machine.interface.Properties.datatype to dao for create 2022-04-26T06:19:10.012Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.Root.datatype to dao for create 2022-04-26T06:19:10.037Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.AssignmentRequirements.datatype to dao for create 2022-04-26T06:19:10.066Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.network.SubnetAssignments.datatype to dao for create 2022-04-26T06:19:10.121Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.network.IPv4SubnetAssignments.datatype to dao for create 2022-04-26T06:19:10.161Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.network.IPv6SubnetAssignments.datatype to dao for create 2022-04-26T06:19:10.204Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.network.NetworkAssignments.datatype to dao for create 2022-04-26T06:19:10.315Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.network.ProviderNetwork.datatype to dao for create 2022-04-26T06:19:10.353Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.network.NetworkFlows.datatype to dao for create 2022-04-26T06:19:10.393Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.network.VlanRequirements.datatype to dao for create 2022-04-26T06:19:10.425Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.network.IpRequirements.datatype to dao for create 2022-04-26T06:19:10.509Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.network.MacAssignments.datatype to dao for create 2022-04-26T06:19:10.541Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.CSProperties.datatype to dao for create 2022-04-26T06:19:10.621Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.NSCapabilities.datatype to dao for create 2022-04-26T06:19:10.712Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.NSSCapabilities.datatype to dao for create 2022-04-26T06:19:10.808Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.ServiceProfile.datatype to dao for create 2022-04-26T06:19:10.902Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.SliceProfile.datatype to dao for create 2022-04-26T06:19:11.003Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.EcompHoming.datatype to dao for create 2022-04-26T06:19:11.030Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.EcompNaming.datatype to dao for create 2022-04-26T06:19:11.054Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.network.MacRequirements.datatype to dao for create 2022-04-26T06:19:11.087Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.machine.subInterface.AddressPairIp.datatype to dao for create 2022-04-26T06:19:11.112Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.machine.subInterface.MacAddress.datatype to dao for create 2022-04-26T06:19:11.130Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.machine.subInterface.Properties.datatype to dao for create 2022-04-26T06:19:11.151Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.machine.subInterface.AddressPair.datatype to dao for create 2022-04-26T06:19:11.183Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.machine.subInterface.AddressPairs.datatype to dao for create 2022-04-26T06:19:11.204Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.Naming.datatype to dao for create 2022-04-26T06:19:11.231Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.flavors.ComputeFlavor.datatype to dao for create 2022-04-26T06:19:11.256Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.flavors.LicenseFlavor.datatype to dao for create 2022-04-26T06:19:11.274Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.flavors.VendorInfo.datatype to dao for create 2022-04-26T06:19:11.300Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.flavors.DeploymentFlavor.datatype to dao for create 2022-04-26T06:19:11.368Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.ImageInfo.datatype to dao for create 2022-04-26T06:19:11.407Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.network.RelatedNetworksAssignments.datatype to dao for create 2022-04-26T06:19:11.432Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfcAdditionalConfigurableProperties.datatype to dao for create 2022-04-26T06:19:11.449Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfcConfigurableProperties.datatype to dao for create 2022-04-26T06:19:11.477Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.RequestedAdditionalCapability.datatype to dao for create 2022-04-26T06:19:11.514Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VirtualMemory.datatype to dao for create 2022-04-26T06:19:11.540Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VirtualCpuPinning.datatype to dao for create 2022-04-26T06:19:11.562Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VirtualCpu.datatype to dao for create 2022-04-26T06:19:11.611Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.LogicalNodeData.datatype to dao for create 2022-04-26T06:19:11.634Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VirtualNetworkInterfaceRequirements.datatype to dao for create 2022-04-26T06:19:11.674Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.injectFile.datatype to dao for create 2022-04-26T06:19:11.699Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.L2AddressData.datatype to dao for create 2022-04-26T06:19:11.721Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.L3AddressData.datatype to dao for create 2022-04-26T06:19:11.751Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.AddressData.datatype to dao for create 2022-04-26T06:19:11.807Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.ConnectivityType.datatype to dao for create 2022-04-26T06:19:11.863Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VlFlavour.datatype to dao for create 2022-04-26T06:19:11.880Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.ext.AddressPairs.datatype to dao for create 2022-04-26T06:19:11.890Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.ext.FloatingIP.datatype to dao for create 2022-04-26T06:19:11.903Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.ext.LocationInfo.datatype to dao for create 2022-04-26T06:19:11.919Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.ext.HostRouteInfo.datatype to dao for create 2022-04-26T06:19:11.931Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.ext.InjectData.datatype to dao for create 2022-04-26T06:19:11.941Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.ext.zte.WatchDog.datatype to dao for create 2022-04-26T06:19:11.953Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.ext.LocalAffinityOrAntiAffinityRule.datatype to dao for create 2022-04-26T06:19:11.970Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VduProfile.datatype to dao for create 2022-04-26T06:19:12.001Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.LinkBitrateRequirements.datatype to dao for create 2022-04-26T06:19:12.025Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.Qos.datatype to dao for create 2022-04-26T06:19:12.049Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.IpAllocationPool.datatype to dao for create 2022-04-26T06:19:12.074Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.L2ProtocolData.datatype to dao for create 2022-04-26T06:19:12.113Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.L3ProtocolData.datatype to dao for create 2022-04-26T06:19:12.159Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VirtualLinkProtocolData.datatype to dao for create 2022-04-26T06:19:12.245Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VlProfile.datatype to dao for create 2022-04-26T06:19:12.371Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VduLevel.datatype to dao for create 2022-04-26T06:19:12.430Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.ScaleInfo.datatype to dao for create 2022-04-26T06:19:12.467Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.ScalingAspect.datatype to dao for create 2022-04-26T06:19:12.502Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.InstantiationLevel.datatype to dao for create 2022-04-26T06:19:12.528Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfInstantiateOperationConfiguration.datatype to dao for create 2022-04-26T06:19:12.547Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfScaleOperationConfiguration.datatype to dao for create 2022-04-26T06:19:12.564Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfScaleToLevelOperationConfiguration.datatype to dao for create 2022-04-26T06:19:12.582Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfHealOperationConfiguration.datatype to dao for create 2022-04-26T06:19:12.604Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfTerminateOperationConfiguration.datatype to dao for create 2022-04-26T06:19:12.625Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfOperateOperationConfiguration.datatype to dao for create 2022-04-26T06:19:12.651Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfChangeFlavourOperationConfiguration.datatype to dao for create 2022-04-26T06:19:12.683Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfProfile.datatype to dao for create 2022-04-26T06:19:12.750Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.UriAuthority.datatype to dao for create 2022-04-26T06:19:12.851Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.UriComponents.datatype to dao for create 2022-04-26T06:19:12.932Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=378eb6b9-ba42-429f-be08-9fdab5f06d43 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:19:12.932Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=378eb6b9-ba42-429f-be08-9fdab5f06d43 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2e5cd7c0 2022-04-26T06:19:12.932Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=378eb6b9-ba42-429f-be08-9fdab5f06d43 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:19:12.933Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=378eb6b9-ba42-429f-be08-9fdab5f06d43 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:19:12.956Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.InterfaceDetails.datatype to dao for create 2022-04-26T06:19:12.992Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VirtualLinkMonitoringParameter.datatype to dao for create 2022-04-26T06:19:13.019Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfcMonitoringParameter.datatype to dao for create 2022-04-26T06:19:13.042Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfChangeExtConnectivityOperationConfiguration.datatype to dao for create 2022-04-26T06:19:13.056Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfMonitoringParameter.datatype to dao for create 2022-04-26T06:19:13.077Z [qtp2085002312-17] WARN o.o.s.b.c.impl.DataTypeImportManager RequestID=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Creating data type tosca.datatypes.nfv.VnfChangeCurrentPackageOperationConfiguration which derived from abstract data type with no properties 2022-04-26T06:19:13.082Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfChangeCurrentPackageOperationConfiguration.datatype to dao for create 2022-04-26T06:19:13.103Z [qtp2085002312-17] WARN o.o.s.b.c.impl.DataTypeImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 Creating data type tosca.datatypes.nfv.VnfCreateSnapshotOperationConfiguration which derived from abstract data type with no properties 2022-04-26T06:19:13.112Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfCreateSnapshotOperationConfiguration.datatype to dao for create 2022-04-26T06:19:13.122Z [qtp2085002312-17] WARN o.o.s.b.c.impl.DataTypeImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 Creating data type tosca.datatypes.nfv.VnfRevertToSnapshotOperationConfiguration which derived from abstract data type with no properties 2022-04-26T06:19:13.128Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfRevertToSnapshotOperationConfiguration.datatype to dao for create 2022-04-26T06:19:13.147Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfLcmOperationsConfiguration.datatype to dao for create 2022-04-26T06:19:13.412Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.CpProtocolData.datatype to dao for create 2022-04-26T06:19:13.464Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfAdditionalConfigurableProperties.datatype to dao for create 2022-04-26T06:19:13.493Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfmInterfaceInfo.datatype to dao for create 2022-04-26T06:19:13.527Z [qtp2085002312-17] WARN o.o.s.b.c.impl.DataTypeImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 Creating data type tosca.datatypes.nfv.OauthServerInfo which derived from abstract data type with no properties 2022-04-26T06:19:13.531Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.OauthServerInfo.datatype to dao for create 2022-04-26T06:19:13.544Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfConfigurableProperties.datatype to dao for create 2022-04-26T06:19:13.641Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfInfoModifiableAttributesExtensions.datatype to dao for create 2022-04-26T06:19:13.664Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfInfoModifiableAttributesMetadata.datatype to dao for create 2022-04-26T06:19:13.679Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfInfoModifiableAttributes.datatype to dao for create 2022-04-26T06:19:13.715Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VirtualLinkBitrateLevel.datatype to dao for create 2022-04-26T06:19:13.747Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfOperationAdditionalParameters.datatype to dao for create 2022-04-26T06:19:13.771Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VirtualFileStorageData.datatype to dao for create 2022-04-26T06:19:13.797Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VirtualObjectStorageData.datatype to dao for create 2022-04-26T06:19:13.815Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VirtualBlockStorageData.datatype to dao for create 2022-04-26T06:19:13.837Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.ChecksumData.datatype to dao for create 2022-04-26T06:19:13.873Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.SwImageData.datatype to dao for create 2022-04-26T06:19:13.928Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.NsVirtualLinkQos.datatype to dao for create 2022-04-26T06:19:13.951Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.ServiceAvailability.datatype to dao for create 2022-04-26T06:19:13.970Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.NsVlProfile.datatype to dao for create 2022-04-26T06:19:14.038Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.NsProfile.datatype to dao for create 2022-04-26T06:19:14.067Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.MciopProfile.datatype to dao for create 2022-04-26T06:19:14.091Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.BootDataVimSpecificProperties.datatype to dao for create 2022-04-26T06:19:14.112Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.KvpData.datatype to dao for create 2022-04-26T06:19:14.129Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.ContentOrFileData.datatype to dao for create 2022-04-26T06:19:14.157Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.BootData.datatype to dao for create 2022-04-26T06:19:14.203Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.ServicePortData.datatype to dao for create 2022-04-26T06:19:14.259Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.AdditionalServiceData.datatype to dao for create 2022-04-26T06:19:14.282Z [qtp2085002312-17] WARN o.o.s.b.c.impl.DataTypeImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 Creating data type tosca.datatypes.nfv.NsOperationAdditionalParameters which derived from abstract data type with no properties 2022-04-26T06:19:14.287Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.NsOperationAdditionalParameters.datatype to dao for create 2022-04-26T06:19:14.304Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.Mask.datatype to dao for create 2022-04-26T06:19:14.347Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.NsMonitoringParameter.datatype to dao for create 2022-04-26T06:19:14.376Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : onap.datatypes.partner.access_details.datatype to dao for create 2022-04-26T06:19:14.457Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : onap.datatypes.partner.bandwidth.datatype to dao for create 2022-04-26T06:19:14.492Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : onap.datatypes.partner.service_details.datatype to dao for create 2022-04-26T06:19:14.625Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : onap.datatypes.partner.sppartner_details.datatype to dao for create 2022-04-26T06:19:14.660Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 send DATA_TYPE : onap.datatypes.ToscaConceptIdentifier.datatype to dao for create 2022-04-26T06:19:14.673Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 all DataTypeDefinition(derivedFrom=null, constraints=null, properties=[PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=onap.datatypes.ToscaConceptIdentifier.datatype.name, type=string, required=true, defaultValue=null, description=null, schema=null, password=false, name=name, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, model=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=null, metadata=null), name=name, constraints=null]], PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=onap.datatypes.ToscaConceptIdentifier.datatype.version, type=string, required=true, defaultValue=null, description=null, schema=null, password=false, name=version, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, model=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=null, metadata=null), name=version, constraints=null]]]) were created successfully!!! 2022-04-26T06:19:15.950Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cd8d8cd3-b533-4f0e-a7da-1be90bd45510 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:19:15.951Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cd8d8cd3-b533-4f0e-a7da-1be90bd45510 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f2975e5 2022-04-26T06:19:15.951Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cd8d8cd3-b533-4f0e-a7da-1be90bd45510 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:19:15.951Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cd8d8cd3-b533-4f0e-a7da-1be90bd45510 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:19:18.849Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20dd94fe-574c-4145-b060-fbe4e2d83b50 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:19:18.850Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20dd94fe-574c-4145-b060-fbe4e2d83b50 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@15c6724e 2022-04-26T06:19:18.850Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20dd94fe-574c-4145-b060-fbe4e2d83b50 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:19:18.850Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20dd94fe-574c-4145-b060-fbe4e2d83b50 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:19:20.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:19:20.380Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:19:22.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56a178fd-7672-4156-b6ee-87b53d88f3fc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:19:22.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56a178fd-7672-4156-b6ee-87b53d88f3fc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@717bdffe 2022-04-26T06:19:22.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56a178fd-7672-4156-b6ee-87b53d88f3fc PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:19:22.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56a178fd-7672-4156-b6ee-87b53d88f3fc PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:19:25.946Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=703595d9-6140-4be8-9155-1c31c9725c69 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:19:25.946Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=703595d9-6140-4be8-9155-1c31c9725c69 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@39d1e4c6 2022-04-26T06:19:25.947Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=703595d9-6140-4be8-9155-1c31c9725c69 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:19:25.947Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=703595d9-6140-4be8-9155-1c31c9725c69 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:19:28.456Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c6aed66-e938-4fb4-99e8-864ddfeeeb04 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:19:28.456Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c6aed66-e938-4fb4-99e8-864ddfeeeb04 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6d033844 2022-04-26T06:19:28.456Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c6aed66-e938-4fb4-99e8-864ddfeeeb04 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:19:28.456Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c6aed66-e938-4fb4-99e8-864ddfeeeb04 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:19:31.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3b09f70-7275-4b1b-a40d-677dbc07f1d0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:19:31.531Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3b09f70-7275-4b1b-a40d-677dbc07f1d0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4b7163a8 2022-04-26T06:19:31.531Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3b09f70-7275-4b1b-a40d-677dbc07f1d0 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:19:31.531Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3b09f70-7275-4b1b-a40d-677dbc07f1d0 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:19:34.433Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d7ad5a12-3e81-4690-96dd-ae236ea631d3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:19:34.433Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d7ad5a12-3e81-4690-96dd-ae236ea631d3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@645ac117 2022-04-26T06:19:34.433Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d7ad5a12-3e81-4690-96dd-ae236ea631d3 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:19:34.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d7ad5a12-3e81-4690-96dd-ae236ea631d3 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:19:35.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:19:35.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:19:37.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80512a08-9361-45d5-a115-cb89574aedc7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:19:37.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80512a08-9361-45d5-a115-cb89574aedc7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@521bfe8e 2022-04-26T06:19:37.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80512a08-9361-45d5-a115-cb89574aedc7 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:19:37.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80512a08-9361-45d5-a115-cb89574aedc7 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:19:40.150Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f07b4fc-c180-4e8c-b8ef-3bb4fcf27c2b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:19:40.150Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f07b4fc-c180-4e8c-b8ef-3bb4fcf27c2b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@26ae2d91 2022-04-26T06:19:40.150Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f07b4fc-c180-4e8c-b8ef-3bb4fcf27c2b PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:19:40.150Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f07b4fc-c180-4e8c-b8ef-3bb4fcf27c2b PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:19:43.560Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b73f2ebf-199a-412d-befc-9219d7bbba84 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:19:43.561Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b73f2ebf-199a-412d-befc-9219d7bbba84 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@660266cf 2022-04-26T06:19:43.561Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b73f2ebf-199a-412d-befc-9219d7bbba84 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:19:43.561Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b73f2ebf-199a-412d-befc-9219d7bbba84 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:19:45.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8cde11db-3c98-4ed3-b381-5f7c4264811c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:19:45.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8cde11db-3c98-4ed3-b381-5f7c4264811c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@54625161 2022-04-26T06:19:45.882Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8cde11db-3c98-4ed3-b381-5f7c4264811c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:19:45.882Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8cde11db-3c98-4ed3-b381-5f7c4264811c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:19:47.633Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=50084 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:18:57.549Z InvocationID=e36e4ded-d2ca-48fa-91ac-8c0ba42d5581 ErrorCode=500 LogTimestamp=2022-04-26T06:19:47.633168Z ResponseDesc=Created Exiting. 2022-04-26T06:19:47.634Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="201" 2022-04-26T06:19:47.673Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c34f7c9f-b963-47e7-9623-336493c70aa6 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/uploadType/capability ServiceInstanceID=null EntryTimestamp=2022-04-26T06:19:47.672998Z InvocationID=ca5ad5b4-1350-462c-a646-5aa4d68e2cd5 LogTimestamp=2022-04-26T06:19:47.673089Z Entering 2022-04-26T06:19:47.674Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c34f7c9f-b963-47e7-9623-336493c70aa6 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/uploadType/capability ServiceInstanceID=null EntryTimestamp=2022-04-26T06:19:47.672998Z InvocationID=ca5ad5b4-1350-462c-a646-5aa4d68e2cd5 LogTimestamp=2022-04-26T06:19:47.673089Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/uploadType/capability HTTP/1.1 2022-04-26T06:19:47.704Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.Root to dao for create 2022-04-26T06:19:47.756Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.Attachment to dao for create 2022-04-26T06:19:47.773Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.Node to dao for create 2022-04-26T06:19:47.847Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.Container to dao for create 2022-04-26T06:19:47.927Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : org.openecomp.capabilities.PortMirroring to dao for create 2022-04-26T06:19:47.949Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.Endpoint to dao for create 2022-04-26T06:19:48.043Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.DatabaseEndpoint to dao for create 2022-04-26T06:19:48.059Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.Endpoint.Public to dao for create 2022-04-26T06:19:48.141Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.Endpoint.Admin to dao for create 2022-04-26T06:19:48.163Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.Endpoint.Database to dao for create 2022-04-26T06:19:48.180Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.OperatingSystem to dao for create 2022-04-26T06:19:48.214Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.Scalable to dao for create 2022-04-26T06:19:48.244Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.network.Bindable to dao for create 2022-04-26T06:19:48.258Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.Container.Docker to dao for create 2022-04-26T06:19:48.319Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.network.Linkable to dao for create 2022-04-26T06:19:48.354Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : org.openecomp.capabilities.AllottedResource to dao for create 2022-04-26T06:19:48.373Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.nfv.Metric to dao for create 2022-04-26T06:19:48.388Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : org.openecomp.capabilities.Metric to dao for create 2022-04-26T06:19:48.462Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : org.openecomp.capabilities.metric.Ceilometer to dao for create 2022-04-26T06:19:48.488Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : org.openecomp.capabilities.metric.SnmpPolling to dao for create 2022-04-26T06:19:48.564Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : org.openecomp.capabilities.metric.SnmpTrap to dao for create 2022-04-26T06:19:48.592Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : org.openecomp.capabilities.Forwarder to dao for create 2022-04-26T06:19:48.658Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : org.openecomp.capabilities.VLANAssignment to dao for create 2022-04-26T06:19:48.680Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : org.openecomp.capabilities.RoutingConfiguration to dao for create 2022-04-26T06:19:48.762Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : org.openecomp.capabilities.FabricConfiguration to dao for create 2022-04-26T06:19:48.855Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.nfv.VirtualStorage to dao for create 2022-04-26T06:19:48.871Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.nfv.VirtualCompute to dao for create 2022-04-26T06:19:49.045Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.nfv.VirtualBindable to dao for create 2022-04-26T06:19:49.055Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d68f378-e891-4157-9ed8-3daabd042b6a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:19:49.055Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d68f378-e891-4157-9ed8-3daabd042b6a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@59002aed 2022-04-26T06:19:49.055Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d68f378-e891-4157-9ed8-3daabd042b6a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:19:49.055Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d68f378-e891-4157-9ed8-3daabd042b6a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:19:49.066Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.nfv.VirtualLinkable to dao for create 2022-04-26T06:19:49.078Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.nfv.ExtVirtualLinkable to dao for create 2022-04-26T06:19:49.147Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.nfv.Forwarder to dao for create 2022-04-26T06:19:49.165Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.nfv.ext.GuestOs to dao for create 2022-04-26T06:19:49.180Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.nfv.ext.ImageFile to dao for create 2022-04-26T06:19:49.259Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.nfv.ext.LocalAttachment to dao for create 2022-04-26T06:19:49.277Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.nfv.ext.Compute.Container.Architecture to dao for create 2022-04-26T06:19:49.363Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.nfv.Forwarding to dao for create 2022-04-26T06:19:49.456Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb all CapabilityTypeDefinition(super=CapabilityTypeDataDefinition [uniqueId=tosca.capabilities.nfv.Forwarding, description=null, type=tosca.capabilities.nfv.Forwarding, validSourceTypes=null, version=null, creationTime=1650953989365, modificationTime=1650953989365], derivedFrom=tosca.capabilities.Root, properties=null, model=null) were created successfully!!! 2022-04-26T06:19:50.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:19:50.377Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:19:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:19:52.686Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:19:52.686Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:19:53.049Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c30e6b13-6e41-4c09-9500-d8176e87ce52 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:19:53.049Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c30e6b13-6e41-4c09-9500-d8176e87ce52 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@e9546f9 2022-04-26T06:19:53.049Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c30e6b13-6e41-4c09-9500-d8176e87ce52 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:19:53.049Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c30e6b13-6e41-4c09-9500-d8176e87ce52 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:19:55.850Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ceac78f-2826-435b-8158-f59f3e3e2193 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:19:55.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ceac78f-2826-435b-8158-f59f3e3e2193 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7b546dfa 2022-04-26T06:19:55.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ceac78f-2826-435b-8158-f59f3e3e2193 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:19:55.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ceac78f-2826-435b-8158-f59f3e3e2193 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:19:57.464Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=9789 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:19:47.674Z InvocationID=f471f872-b4f6-434b-b322-d8028ea5ad63 LogTimestamp=2022-04-26T06:19:57.463971Z ResponseDesc=Created Exiting. 2022-04-26T06:19:57.465Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/capability HTTP/1.1 SC="201" 2022-04-26T06:19:57.485Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=e9fe776f-6f26-402b-b985-6b3daf734296 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/uploadType/relationship ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:19:57.484667Z InvocationID=72c4cc25-17f2-42e8-8dc2-762b2bb7e409 LogTimestamp=2022-04-26T06:19:57.484773Z Entering 2022-04-26T06:19:57.485Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=e9fe776f-6f26-402b-b985-6b3daf734296 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/uploadType/relationship ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:19:57.484667Z InvocationID=72c4cc25-17f2-42e8-8dc2-762b2bb7e409 LogTimestamp=2022-04-26T06:19:57.484773Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/uploadType/relationship HTTP/1.1 2022-04-26T06:19:57.504Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:19:57.485Z InvocationID=f4a476b7-cb58-45cd-bdda-e7334612a634 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.Root to dao for create 2022-04-26T06:19:58.060Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:19:57.485Z InvocationID=f4a476b7-cb58-45cd-bdda-e7334612a634 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.ConnectsTo to dao for create 2022-04-26T06:19:58.453Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=716f959a-c532-4e68-ae3b-0ec2e81d2879 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:19:58.454Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=716f959a-c532-4e68-ae3b-0ec2e81d2879 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3d7bc2a8 2022-04-26T06:19:58.454Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=716f959a-c532-4e68-ae3b-0ec2e81d2879 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:19:58.454Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=716f959a-c532-4e68-ae3b-0ec2e81d2879 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:19:59.450Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:19:57.485Z InvocationID=f4a476b7-cb58-45cd-bdda-e7334612a634 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.RoutesTo to dao for create 2022-04-26T06:20:01.047Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:19:57.485Z InvocationID=f4a476b7-cb58-45cd-bdda-e7334612a634 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.AttachesTo to dao for create 2022-04-26T06:20:01.735Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fad3256f-82af-4100-9bcc-e0e06c040790 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:20:01.735Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fad3256f-82af-4100-9bcc-e0e06c040790 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6574d0a8 2022-04-26T06:20:01.735Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fad3256f-82af-4100-9bcc-e0e06c040790 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:20:01.736Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fad3256f-82af-4100-9bcc-e0e06c040790 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:20:03.652Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:19:57.485Z InvocationID=f4a476b7-cb58-45cd-bdda-e7334612a634 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.HostedOn to dao for create 2022-04-26T06:20:04.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da4e4c8c-c3aa-44d8-b983-8e80560ffd7f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:20:04.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da4e4c8c-c3aa-44d8-b983-8e80560ffd7f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4172f071 2022-04-26T06:20:04.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da4e4c8c-c3aa-44d8-b983-8e80560ffd7f PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:20:04.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da4e4c8c-c3aa-44d8-b983-8e80560ffd7f PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:20:04.928Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:19:57.485Z InvocationID=f4a476b7-cb58-45cd-bdda-e7334612a634 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.DependsOn to dao for create 2022-04-26T06:20:05.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:20:05.379Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:20:05.642Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:19:57.485Z InvocationID=f4a476b7-cb58-45cd-bdda-e7334612a634 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.network.LinksTo to dao for create 2022-04-26T06:20:06.645Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:19:57.485Z InvocationID=f4a476b7-cb58-45cd-bdda-e7334612a634 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.network.BindsTo to dao for create 2022-04-26T06:20:07.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9a50a5a-148e-4aa6-90b1-ba8f5ec9216d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:20:07.531Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9a50a5a-148e-4aa6-90b1-ba8f5ec9216d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@19116e18 2022-04-26T06:20:07.531Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9a50a5a-148e-4aa6-90b1-ba8f5ec9216d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:20:07.531Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9a50a5a-148e-4aa6-90b1-ba8f5ec9216d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:20:07.641Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:20:07.842Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:19:57.485Z InvocationID=f4a476b7-cb58-45cd-bdda-e7334612a634 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : org.openecomp.relationships.AttachesTo to dao for create 2022-04-26T06:20:08.852Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:19:57.485Z InvocationID=f4a476b7-cb58-45cd-bdda-e7334612a634 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : org.openecomp.relationships.VolumeAttachesTo to dao for create 2022-04-26T06:20:09.634Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:19:57.485Z InvocationID=f4a476b7-cb58-45cd-bdda-e7334612a634 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : org.openecomp.relationships.ForwardsTo to dao for create 2022-04-26T06:20:09.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=384e5399-6fcd-484e-bc82-46fe43b840b9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:20:09.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=384e5399-6fcd-484e-bc82-46fe43b840b9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7a9c9110 2022-04-26T06:20:09.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=384e5399-6fcd-484e-bc82-46fe43b840b9 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:20:09.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=384e5399-6fcd-484e-bc82-46fe43b840b9 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:20:10.262Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:19:57.485Z InvocationID=f4a476b7-cb58-45cd-bdda-e7334612a634 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : org.openecomp.relationships.AssignsTo to dao for create 2022-04-26T06:20:10.798Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:19:57.485Z InvocationID=f4a476b7-cb58-45cd-bdda-e7334612a634 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : org.openecomp.relationships.RoutesTo to dao for create 2022-04-26T06:20:11.376Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:19:57.485Z InvocationID=f4a476b7-cb58-45cd-bdda-e7334612a634 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.nfv.VirtualBindsTo to dao for create 2022-04-26T06:20:11.912Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:19:57.485Z InvocationID=f4a476b7-cb58-45cd-bdda-e7334612a634 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.nfv.VirtualLinksTo to dao for create 2022-04-26T06:20:12.453Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:19:57.485Z InvocationID=f4a476b7-cb58-45cd-bdda-e7334612a634 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.nfv.AttachesTo to dao for create 2022-04-26T06:20:12.788Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=329e3fa8-cc7f-42a1-93fe-d463fdd18e56 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:20:12.788Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=329e3fa8-cc7f-42a1-93fe-d463fdd18e56 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@39aeff7b 2022-04-26T06:20:12.788Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=329e3fa8-cc7f-42a1-93fe-d463fdd18e56 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:20:12.788Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=329e3fa8-cc7f-42a1-93fe-d463fdd18e56 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:20:12.983Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:19:57.485Z InvocationID=f4a476b7-cb58-45cd-bdda-e7334612a634 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.nfv.Vdu.AttachedTo to dao for create 2022-04-26T06:20:13.553Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:19:57.485Z InvocationID=f4a476b7-cb58-45cd-bdda-e7334612a634 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.nfv.ForwardTo to dao for create 2022-04-26T06:20:14.087Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:19:57.485Z InvocationID=f4a476b7-cb58-45cd-bdda-e7334612a634 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.nfv.ContainerBindsTo to dao for create 2022-04-26T06:20:14.625Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:19:57.485Z InvocationID=f4a476b7-cb58-45cd-bdda-e7334612a634 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb all RelationshipTypeDefinition(super=RelationshipTypeDataDefinition [uniqueId=tosca.relationships.nfv.ContainerBindsTo, description=Represents an association relationship between Vdu.OsContainerGroup and VduCp node types, type=tosca.relationships.nfv.ContainerBindsTo, validSourceTypes=null, version=null, originUI=null, creationTime=1650954014090, modificationTime=1650954014090], derivedFrom=tosca.relationships.DependsOn, properties=null, model=null) were created successfully!!! 2022-04-26T06:20:14.632Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=17146 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:19:57.485Z InvocationID=f4a476b7-cb58-45cd-bdda-e7334612a634 LogTimestamp=2022-04-26T06:20:14.631987Z ResponseDesc=Created Exiting. 2022-04-26T06:20:14.632Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/relationship HTTP/1.1 SC="201" 2022-04-26T06:20:14.646Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=03676fa9-423a-48b8-9904-7eef10a094b0 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/uploadType/interfaceLifecycle ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:20:14.645974Z InvocationID=1ae295dc-ee64-472c-b72a-9ca6f2d87aa9 LogTimestamp=2022-04-26T06:20:14.646076Z Entering 2022-04-26T06:20:14.646Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=03676fa9-423a-48b8-9904-7eef10a094b0 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/uploadType/interfaceLifecycle ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:20:14.645974Z InvocationID=1ae295dc-ee64-472c-b72a-9ca6f2d87aa9 LogTimestamp=2022-04-26T06:20:14.646076Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/uploadType/interfaceLifecycle HTTP/1.1 2022-04-26T06:20:14.662Z [qtp2085002312-16] INFO o.o.s.b.c.i.InterfaceLifecycleTypeImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2022-04-26T06:20:14.647Z InvocationID=f19e5fd1-5ac7-455f-8b7a-31bb3de19b28 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send interfaceDefinition tosca.interfaces.Root to dao for create 2022-04-26T06:20:15.184Z [qtp2085002312-16] INFO o.o.s.b.c.i.InterfaceLifecycleTypeImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2022-04-26T06:20:14.647Z InvocationID=f19e5fd1-5ac7-455f-8b7a-31bb3de19b28 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send interfaceDefinition tosca.interfaces.nfv.vnf.lifecycle.Nfv to dao for create 2022-04-26T06:20:15.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55e83af2-07c5-4fa4-bc2c-99e9fdfb5fe1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:20:15.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55e83af2-07c5-4fa4-bc2c-99e9fdfb5fe1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6f096147 2022-04-26T06:20:15.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55e83af2-07c5-4fa4-bc2c-99e9fdfb5fe1 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:20:15.785Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55e83af2-07c5-4fa4-bc2c-99e9fdfb5fe1 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:20:16.811Z [qtp2085002312-16] INFO o.o.s.b.c.i.InterfaceLifecycleTypeImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2022-04-26T06:20:14.647Z InvocationID=f19e5fd1-5ac7-455f-8b7a-31bb3de19b28 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send interfaceDefinition tosca.interfaces.node.lifecycle.Standard to dao for create 2022-04-26T06:20:17.404Z [qtp2085002312-16] INFO o.o.s.b.c.i.InterfaceLifecycleTypeImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2022-04-26T06:20:14.647Z InvocationID=f19e5fd1-5ac7-455f-8b7a-31bb3de19b28 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send interfaceDefinition tosca.interfaces.relationship.Configure to dao for create 2022-04-26T06:20:18.076Z [qtp2085002312-16] INFO o.o.s.b.c.i.InterfaceLifecycleTypeImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2022-04-26T06:20:14.647Z InvocationID=f19e5fd1-5ac7-455f-8b7a-31bb3de19b28 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send interfaceDefinition tosca.interfaces.nfv.Vnflcm to dao for create 2022-04-26T06:20:18.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc33232f-ae9a-4ac7-9151-00f49e64d579 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:20:18.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc33232f-ae9a-4ac7-9151-00f49e64d579 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7d7ab310 2022-04-26T06:20:18.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc33232f-ae9a-4ac7-9151-00f49e64d579 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:20:18.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc33232f-ae9a-4ac7-9151-00f49e64d579 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:20:19.273Z [qtp2085002312-16] INFO o.o.s.b.c.i.InterfaceLifecycleTypeImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2022-04-26T06:20:14.647Z InvocationID=f19e5fd1-5ac7-455f-8b7a-31bb3de19b28 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lifecycle types were created successfully!!! 2022-04-26T06:20:19.288Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=4640 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2022-04-26T06:20:14.647Z InvocationID=f19e5fd1-5ac7-455f-8b7a-31bb3de19b28 LogTimestamp=2022-04-26T06:20:19.287716Z ResponseDesc=Created Exiting. 2022-04-26T06:20:19.288Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/interfaceLifecycle HTTP/1.1 SC="201" 2022-04-26T06:20:19.298Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=eb4c3279-885b-4a19-8e4f-fc4fc26b367d InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/uploadType/categories ServiceInstanceID=null EntryTimestamp=2022-04-26T06:20:19.297934Z InvocationID=8ed8d38f-dcfc-4ea6-b389-a3bbb2cf8e23 LogTimestamp=2022-04-26T06:20:19.298003Z Entering 2022-04-26T06:20:19.298Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=eb4c3279-885b-4a19-8e4f-fc4fc26b367d InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/uploadType/categories ServiceInstanceID=null EntryTimestamp=2022-04-26T06:20:19.297934Z InvocationID=8ed8d38f-dcfc-4ea6-b389-a3bbb2cf8e23 LogTimestamp=2022-04-26T06:20:19.298003Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/uploadType/categories HTTP/1.1 2022-04-26T06:20:20.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:20:20.374Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:20:22.747Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f15bef6-b328-42fa-9864-9d86a91e43b3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:20:22.747Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f15bef6-b328-42fa-9864-9d86a91e43b3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@63f0a9be 2022-04-26T06:20:22.748Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f15bef6-b328-42fa-9864-9d86a91e43b3 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:20:22.748Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f15bef6-b328-42fa-9864-9d86a91e43b3 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:20:25.361Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18a51c86-cc67-49ad-9e97-8182464b9427 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:20:25.361Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18a51c86-cc67-49ad-9e97-8182464b9427 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2d6bd79c 2022-04-26T06:20:25.361Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18a51c86-cc67-49ad-9e97-8182464b9427 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:20:25.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18a51c86-cc67-49ad-9e97-8182464b9427 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:20:28.559Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=170b7c2e-e725-4ba7-9778-87c60e887141 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:20:28.559Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=170b7c2e-e725-4ba7-9778-87c60e887141 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9cc982f 2022-04-26T06:20:28.559Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=170b7c2e-e725-4ba7-9778-87c60e887141 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:20:28.560Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=170b7c2e-e725-4ba7-9778-87c60e887141 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:20:31.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc6aab5a-250a-4f04-a58d-98a4803dd300 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:20:31.634Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc6aab5a-250a-4f04-a58d-98a4803dd300 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@21064835 2022-04-26T06:20:31.634Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc6aab5a-250a-4f04-a58d-98a4803dd300 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:20:31.634Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc6aab5a-250a-4f04-a58d-98a4803dd300 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:20:34.741Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2d60399a-bbc9-4113-94e8-92fdcb705ee2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:20:34.741Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2d60399a-bbc9-4113-94e8-92fdcb705ee2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@475da8f2 2022-04-26T06:20:34.741Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2d60399a-bbc9-4113-94e8-92fdcb705ee2 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:20:34.741Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2d60399a-bbc9-4113-94e8-92fdcb705ee2 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:20:35.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:20:35.380Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:20:37.731Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7a6d9f1-fdc8-4c95-a861-7a12f573c19e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:20:37.731Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7a6d9f1-fdc8-4c95-a861-7a12f573c19e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@347950b2 2022-04-26T06:20:37.731Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7a6d9f1-fdc8-4c95-a861-7a12f573c19e PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:20:37.732Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7a6d9f1-fdc8-4c95-a861-7a12f573c19e PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:20:39.228Z [ApplicationDataTypeCacheThread-1] INFO o.o.s.b.m.c.ApplicationDataTypeCache RequestID=null Detected changes in the data types, updating the data type cache. 2022-04-26T06:20:40.433Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e72384fb-53ac-4ac5-9d25-263f3e87626d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:20:40.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e72384fb-53ac-4ac5-9d25-263f3e87626d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5919bbf2 2022-04-26T06:20:40.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e72384fb-53ac-4ac5-9d25-263f3e87626d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:20:40.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e72384fb-53ac-4ac5-9d25-263f3e87626d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:20:42.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab6483fa-38b0-4d41-a28a-ecb8c269fffa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:20:42.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab6483fa-38b0-4d41-a28a-ecb8c269fffa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4a4d1704 2022-04-26T06:20:42.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab6483fa-38b0-4d41-a28a-ecb8c269fffa PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:20:42.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab6483fa-38b0-4d41-a28a-ecb8c269fffa PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:20:44.964Z [ApplicationDataTypeCacheThread-1] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServiceName=ReplaceDataTypesCache TargetServiceName=null ErrorCode=500 TargetEntity=ReplaceDataTypesCache Internal flow error. Operation: Succeed to replace the data types cache 2022-04-26T06:20:45.841Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b99d8a05-d92d-433f-ac3d-1d8d42efc3bc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:20:45.842Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b99d8a05-d92d-433f-ac3d-1d8d42efc3bc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@607d3c87 2022-04-26T06:20:45.842Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b99d8a05-d92d-433f-ac3d-1d8d42efc3bc PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:20:45.842Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b99d8a05-d92d-433f-ac3d-1d8d42efc3bc PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:20:48.790Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5d72a6b-8cb6-400b-ba42-e0178d0123fc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:20:48.790Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5d72a6b-8cb6-400b-ba42-e0178d0123fc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1e7f0cfa 2022-04-26T06:20:48.790Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5d72a6b-8cb6-400b-ba42-e0178d0123fc PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:20:48.791Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5d72a6b-8cb6-400b-ba42-e0178d0123fc PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:20:50.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:20:50.379Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:20:52.452Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a28c50d9-35d8-4711-bc30-0c60ff18a296 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:20:52.453Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a28c50d9-35d8-4711-bc30-0c60ff18a296 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@a2df2ec 2022-04-26T06:20:52.453Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a28c50d9-35d8-4711-bc30-0c60ff18a296 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:20:52.453Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a28c50d9-35d8-4711-bc30-0c60ff18a296 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:20:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:20:52.686Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:20:52.686Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:20:55.452Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2075f21-557b-4f5d-b3a7-b59b119a674f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:20:55.452Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2075f21-557b-4f5d-b3a7-b59b119a674f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@28fa319a 2022-04-26T06:20:55.452Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2075f21-557b-4f5d-b3a7-b59b119a674f PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:20:55.452Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2075f21-557b-4f5d-b3a7-b59b119a674f PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:20:58.547Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25e62e87-e4a1-45fd-9b4f-e221cc8c6386 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:20:58.547Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25e62e87-e4a1-45fd-9b4f-e221cc8c6386 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@21eaa906 2022-04-26T06:20:58.548Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25e62e87-e4a1-45fd-9b4f-e221cc8c6386 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:20:58.548Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25e62e87-e4a1-45fd-9b4f-e221cc8c6386 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:21:00.746Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=41448 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories EntryTimestamp=2022-04-26T06:20:19.298Z InvocationID=453baba5-34c6-4461-b849-41abde64753e LogTimestamp=2022-04-26T06:21:00.746333Z ResponseDesc=Created Exiting. 2022-04-26T06:21:00.747Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/categories HTTP/1.1 SC="201" 2022-04-26T06:21:00.761Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=451db034-2646-4d33-b585-b7603659b442 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/uploadType/artifactTypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:00.761409Z InvocationID=52a60c91-74ff-4d72-ba6e-c9c131c23ea7 LogTimestamp=2022-04-26T06:21:00.761546Z Entering 2022-04-26T06:21:00.762Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=451db034-2646-4d33-b585-b7603659b442 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/uploadType/artifactTypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:00.761409Z InvocationID=52a60c91-74ff-4d72-ba6e-c9c131c23ea7 LogTimestamp=2022-04-26T06:21:00.761546Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/uploadType/artifactTypes HTTP/1.1 2022-04-26T06:21:01.442Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a954c1df-e438-430d-9cd5-643d39047fda PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:21:01.442Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a954c1df-e438-430d-9cd5-643d39047fda PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4794791d 2022-04-26T06:21:01.442Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a954c1df-e438-430d-9cd5-643d39047fda PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:21:01.442Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a954c1df-e438-430d-9cd5-643d39047fda PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:21:04.334Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=104033ba-d4b2-4ee5-aa3b-7a222cf09777 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:21:04.334Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=104033ba-d4b2-4ee5-aa3b-7a222cf09777 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@224b4991 2022-04-26T06:21:04.335Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=104033ba-d4b2-4ee5-aa3b-7a222cf09777 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:21:04.335Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=104033ba-d4b2-4ee5-aa3b-7a222cf09777 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:21:05.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:21:05.377Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:21:07.332Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b1f9e7c-f323-4308-b6a0-3d10cc8d9da9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:21:07.333Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b1f9e7c-f323-4308-b6a0-3d10cc8d9da9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5bc4caa3 2022-04-26T06:21:07.333Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b1f9e7c-f323-4308-b6a0-3d10cc8d9da9 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:21:07.333Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b1f9e7c-f323-4308-b6a0-3d10cc8d9da9 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:21:07.657Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:21:08.467Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=7705 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/artifactTypes EntryTimestamp=2022-04-26T06:21:00.762Z InvocationID=c1ffad37-9d5d-4de6-82c3-8d125feec17e LogTimestamp=2022-04-26T06:21:08.467088Z ResponseDesc=Created Exiting. 2022-04-26T06:21:08.468Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/artifactTypes HTTP/1.1 SC="201" 2022-04-26T06:21:08.483Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=2a2e037e-6e76-4b9b-9ac8-ede43fe99f67 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:08.483274Z InvocationID=815668e7-d751-4937-933a-b1c04dca95e1 LogTimestamp=2022-04-26T06:21:08.483375Z Entering 2022-04-26T06:21:08.484Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=2a2e037e-6e76-4b9b-9ac8-ede43fe99f67 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:08.483274Z InvocationID=815668e7-d751-4937-933a-b1c04dca95e1 LogTimestamp=2022-04-26T06:21:08.483375Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:21:09.015Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:08.485Z InvocationID=8f45286f-67e1-407e-a914-3b288fa103a0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:09.016Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:08.485Z InvocationID=8f45286f-67e1-407e-a914-3b288fa103a0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:09.016Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:08.485Z InvocationID=8f45286f-67e1-407e-a914-3b288fa103a0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:09.016Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:08.485Z InvocationID=8f45286f-67e1-407e-a914-3b288fa103a0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:09.040Z [qtp2085002312-12] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Create Resource - validateLifecycleTypesCreate ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:21:08.485Z InvocationID=8f45286f-67e1-407e-a914-3b288fa103a0 ErrorCode=400 Interface tosca.interfaces.node.lifecycle.standard required is missing 2022-04-26T06:21:09.040Z [qtp2085002312-12] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Create Resource - validateLifecycleTypesCreate ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:21:08.485Z InvocationID=8f45286f-67e1-407e-a914-3b288fa103a0 ErrorCode=500 Operation towards database failed. 2022-04-26T06:21:09.040Z [qtp2085002312-12] WARN o.o.s.e.AbstractSdncException RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:21:08.485Z InvocationID=8f45286f-67e1-407e-a914-3b288fa103a0 ErrorCode=900 2022-04-26T06:21:09.060Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Root" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "" INVARIANT_UUID = "" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "400" DESC = "SVC4134: Error: Invalid Content. Missing interface life-cycle type." TOSCA_NODE_TYPE = "tosca.nodes.Root" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:21:08.485Z InvocationID=8f45286f-67e1-407e-a914-3b288fa103a0 ErrorCode=900 lock resource with name Root 2022-04-26T06:21:09.961Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=428fc521-e0b4-439e-9fac-02fac7e31a95 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:21:09.961Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=428fc521-e0b4-439e-9fac-02fac7e31a95 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f92fa44 2022-04-26T06:21:09.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=428fc521-e0b4-439e-9fac-02fac7e31a95 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:21:09.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=428fc521-e0b4-439e-9fac-02fac7e31a95 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:21:10.574Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Root" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e77bea56-f959-401d-b1bf-8bb74ab64c5a" INVARIANT_UUID = "60d16850-a83a-4f6b-965b-791ca249550b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.Root" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e77bea56-f959-401d-b1bf-8bb74ab64c5a TargetServiceName=null EntryTimestamp=2022-04-26T06:21:08.485Z InvocationID=8f45286f-67e1-407e-a914-3b288fa103a0 ErrorCode=900 lock resource with id a5b0efb2-943f-4835-aeed-e12849fee5e0 2022-04-26T06:21:12.104Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Root" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e77bea56-f959-401d-b1bf-8bb74ab64c5a" INVARIANT_UUID = "60d16850-a83a-4f6b-965b-791ca249550b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a5b0efb2-943f-4835-aeed-e12849fee5e0.assettoscatemplate" CURR_ARTIFACT_UUID = "a5b0efb2-943f-4835-aeed-e12849fee5e0.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Root-template.yml,0,a5b0efb2-943f-4835-aeed-e12849fee5e0.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=3619 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=73654c90-5a34-4e12-8a6a-09f716564aa4 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:08.485Z InvocationID=8f45286f-67e1-407e-a914-3b288fa103a0 ErrorCode=900 LogTimestamp=2022-04-26T06:21:12.104399Z ResponseDesc=Created Exiting. 2022-04-26T06:21:12.105Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:21:12.117Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=3b449bb0-72fb-4179-b71f-eb46e1fe5794 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:12.117126Z InvocationID=9a368172-a300-4f77-831e-e58034a18180 LogTimestamp=2022-04-26T06:21:12.117237Z Entering 2022-04-26T06:21:12.117Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=3b449bb0-72fb-4179-b71f-eb46e1fe5794 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:12.117126Z InvocationID=9a368172-a300-4f77-831e-e58034a18180 LogTimestamp=2022-04-26T06:21:12.117237Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:21:12.684Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:12.118Z InvocationID=70fd9d6a-b140-406f-940f-1f8c4d36df59 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:12.685Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:12.118Z InvocationID=70fd9d6a-b140-406f-940f-1f8c4d36df59 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:12.685Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:12.118Z InvocationID=70fd9d6a-b140-406f-940f-1f8c4d36df59 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:12.685Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:12.118Z InvocationID=70fd9d6a-b140-406f-940f-1f8c4d36df59 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:13.051Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:12.118Z InvocationID=70fd9d6a-b140-406f-940f-1f8c4d36df59 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Compute 2022-04-26T06:21:13.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=438c67b8-7663-4f5d-accc-1c1fe92ad88a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:21:13.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=438c67b8-7663-4f5d-accc-1c1fe92ad88a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4baa223a 2022-04-26T06:21:13.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=438c67b8-7663-4f5d-accc-1c1fe92ad88a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:21:13.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=438c67b8-7663-4f5d-accc-1c1fe92ad88a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:21:13.276Z [qtp2085002312-15] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a1f8df9f-38ca-428d-baa7-c77fcb1837ff" INVARIANT_UUID = "f25e98f6-31ae-4d86-a38b-5f40cb115c71" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.Compute" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a1f8df9f-38ca-428d-baa7-c77fcb1837ff EntryTimestamp=2022-04-26T06:21:12.118Z InvocationID=70fd9d6a-b140-406f-940f-1f8c4d36df59 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:21:13.277Z [qtp2085002312-15] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a1f8df9f-38ca-428d-baa7-c77fcb1837ff" INVARIANT_UUID = "f25e98f6-31ae-4d86-a38b-5f40cb115c71" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.Compute" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a1f8df9f-38ca-428d-baa7-c77fcb1837ff TargetServiceName=null EntryTimestamp=2022-04-26T06:21:12.118Z InvocationID=70fd9d6a-b140-406f-940f-1f8c4d36df59 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:21:13.408Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a1f8df9f-38ca-428d-baa7-c77fcb1837ff" INVARIANT_UUID = "f25e98f6-31ae-4d86-a38b-5f40cb115c71" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.Compute" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a1f8df9f-38ca-428d-baa7-c77fcb1837ff TargetServiceName=null EntryTimestamp=2022-04-26T06:21:12.118Z InvocationID=70fd9d6a-b140-406f-940f-1f8c4d36df59 ErrorCode=500 lock resource with id 99afa7fe-97db-4a77-a3f7-1ee331794e39 2022-04-26T06:21:14.578Z [qtp2085002312-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a1f8df9f-38ca-428d-baa7-c77fcb1837ff" INVARIANT_UUID = "f25e98f6-31ae-4d86-a38b-5f40cb115c71" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "99afa7fe-97db-4a77-a3f7-1ee331794e39.assettoscatemplate" CURR_ARTIFACT_UUID = "99afa7fe-97db-4a77-a3f7-1ee331794e39.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Compute-template.yml,0,99afa7fe-97db-4a77-a3f7-1ee331794e39.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0487c523-6764-43f6-bc1b-97e22fcdc9dd TargetServiceName=null EntryTimestamp=2022-04-26T06:21:12.118Z InvocationID=70fd9d6a-b140-406f-940f-1f8c4d36df59 ErrorCode=500 Starting to create Csar for component Compute 2022-04-26T06:21:15.126Z [qtp2085002312-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a1f8df9f-38ca-428d-baa7-c77fcb1837ff" INVARIANT_UUID = "f25e98f6-31ae-4d86-a38b-5f40cb115c71" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "99afa7fe-97db-4a77-a3f7-1ee331794e39.assettoscatemplate" CURR_ARTIFACT_UUID = "99afa7fe-97db-4a77-a3f7-1ee331794e39.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Compute-template.yml,0,99afa7fe-97db-4a77-a3f7-1ee331794e39.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0487c523-6764-43f6-bc1b-97e22fcdc9dd TargetServiceName=null EntryTimestamp=2022-04-26T06:21:12.118Z InvocationID=70fd9d6a-b140-406f-940f-1f8c4d36df59 ErrorCode=500 Ended create Csar for component Compute 2022-04-26T06:21:15.131Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a1f8df9f-38ca-428d-baa7-c77fcb1837ff" INVARIANT_UUID = "f25e98f6-31ae-4d86-a38b-5f40cb115c71" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "99afa7fe-97db-4a77-a3f7-1ee331794e39.assettoscatemplate" CURR_ARTIFACT_UUID = "99afa7fe-97db-4a77-a3f7-1ee331794e39.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Compute-template.yml,0,99afa7fe-97db-4a77-a3f7-1ee331794e39.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0487c523-6764-43f6-bc1b-97e22fcdc9dd TargetServiceName=null EntryTimestamp=2022-04-26T06:21:12.118Z InvocationID=70fd9d6a-b140-406f-940f-1f8c4d36df59 ErrorCode=500 lock resource with id 99afa7fe-97db-4a77-a3f7-1ee331794e39 2022-04-26T06:21:15.782Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a1f8df9f-38ca-428d-baa7-c77fcb1837ff" INVARIANT_UUID = "f25e98f6-31ae-4d86-a38b-5f40cb115c71" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "99afa7fe-97db-4a77-a3f7-1ee331794e39.assettoscacsar" CURR_ARTIFACT_UUID = "99afa7fe-97db-4a77-a3f7-1ee331794e39.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Compute-csar.csar,0,99afa7fe-97db-4a77-a3f7-1ee331794e39.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=3664 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=dc75a52d-58a1-45a8-8d2c-f476bc31178e TargetServiceName=null EntryTimestamp=2022-04-26T06:21:12.118Z InvocationID=70fd9d6a-b140-406f-940f-1f8c4d36df59 ErrorCode=500 LogTimestamp=2022-04-26T06:21:15.782161Z ResponseDesc=Created Exiting. 2022-04-26T06:21:15.783Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:21:15.802Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=eff1d46d-b3cc-4858-83ca-dfc04d6ce333 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:15.802388Z InvocationID=076729ca-118f-473b-a749-1dbe0d43274e LogTimestamp=2022-04-26T06:21:15.802543Z Entering 2022-04-26T06:21:15.803Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=eff1d46d-b3cc-4858-83ca-dfc04d6ce333 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:15.802388Z InvocationID=076729ca-118f-473b-a749-1dbe0d43274e LogTimestamp=2022-04-26T06:21:15.802543Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:21:15.844Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=376feeea-c23b-44b8-9dbe-f99f03ece49c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:21:15.844Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=376feeea-c23b-44b8-9dbe-f99f03ece49c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3355fd1f 2022-04-26T06:21:15.844Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=376feeea-c23b-44b8-9dbe-f99f03ece49c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:21:15.845Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=376feeea-c23b-44b8-9dbe-f99f03ece49c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:21:16.071Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:15.804Z InvocationID=306354c0-74d6-4e84-be54-7926ea25f82e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:16.071Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:15.804Z InvocationID=306354c0-74d6-4e84-be54-7926ea25f82e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:16.072Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:15.804Z InvocationID=306354c0-74d6-4e84-be54-7926ea25f82e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:16.072Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:15.804Z InvocationID=306354c0-74d6-4e84-be54-7926ea25f82e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:16.079Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:15.804Z InvocationID=306354c0-74d6-4e84-be54-7926ea25f82e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Softwarecomponent 2022-04-26T06:21:16.213Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "SoftwareComponent" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "5f9ebe17-1319-4eaa-aa5a-5eaff59fa7bb" INVARIANT_UUID = "d410d956-1515-4ee7-9f4b-e393b4d3ce6a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.SoftwareComponent" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5f9ebe17-1319-4eaa-aa5a-5eaff59fa7bb EntryTimestamp=2022-04-26T06:21:15.804Z InvocationID=306354c0-74d6-4e84-be54-7926ea25f82e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 976d9ad0-1ac9-4dce-9b70-babd144a37db 2022-04-26T06:21:17.383Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "SoftwareComponent" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "5f9ebe17-1319-4eaa-aa5a-5eaff59fa7bb" INVARIANT_UUID = "d410d956-1515-4ee7-9f4b-e393b4d3ce6a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "976d9ad0-1ac9-4dce-9b70-babd144a37db.assettoscatemplate" CURR_ARTIFACT_UUID = "976d9ad0-1ac9-4dce-9b70-babd144a37db.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Softwarecomponent-template.yml,0,976d9ad0-1ac9-4dce-9b70-babd144a37db.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=1578 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=32d0f317-0ae4-4a95-843f-dd69b0b660e5 EntryTimestamp=2022-04-26T06:21:15.804Z InvocationID=306354c0-74d6-4e84-be54-7926ea25f82e LogTimestamp=2022-04-26T06:21:17.382997Z ResponseDesc=Created Exiting. 2022-04-26T06:21:17.383Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:21:17.394Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=bb98ec4e-dc9e-4dbb-909e-3f5fb65178ae InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:17.394595Z InvocationID=5f234b52-1324-4fab-8a18-0d9b477ab4bd LogTimestamp=2022-04-26T06:21:17.394675Z Entering 2022-04-26T06:21:17.395Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=bb98ec4e-dc9e-4dbb-909e-3f5fb65178ae InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:17.394595Z InvocationID=5f234b52-1324-4fab-8a18-0d9b477ab4bd LogTimestamp=2022-04-26T06:21:17.394675Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:21:17.688Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:17.395Z InvocationID=78034ea6-1280-4605-8931-543b6fb864bf PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:17.688Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:17.395Z InvocationID=78034ea6-1280-4605-8931-543b6fb864bf PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:17.688Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:17.395Z InvocationID=78034ea6-1280-4605-8931-543b6fb864bf PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:17.688Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:17.395Z InvocationID=78034ea6-1280-4605-8931-543b6fb864bf PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:17.852Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:17.395Z InvocationID=78034ea6-1280-4605-8931-543b6fb864bf PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Webserver 2022-04-26T06:21:18.075Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "WebServer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a71c1132-1e2b-405d-86e3-26a442801039" INVARIANT_UUID = "7afba0ce-6e3f-4f17-a1ca-cb47d63a76a1" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.WebServer" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a71c1132-1e2b-405d-86e3-26a442801039 EntryTimestamp=2022-04-26T06:21:17.395Z InvocationID=78034ea6-1280-4605-8931-543b6fb864bf PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 4af9dd3d-4cbe-4bef-9df5-8c1f6a146cb8 2022-04-26T06:21:18.989Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "WebServer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a71c1132-1e2b-405d-86e3-26a442801039" INVARIANT_UUID = "7afba0ce-6e3f-4f17-a1ca-cb47d63a76a1" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "4af9dd3d-4cbe-4bef-9df5-8c1f6a146cb8.assettoscatemplate" CURR_ARTIFACT_UUID = "4af9dd3d-4cbe-4bef-9df5-8c1f6a146cb8.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Webserver-template.yml,0,4af9dd3d-4cbe-4bef-9df5-8c1f6a146cb8.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=1594 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b3498a61-3fa4-47c3-a4cf-fb44ab1927d8 EntryTimestamp=2022-04-26T06:21:17.395Z InvocationID=78034ea6-1280-4605-8931-543b6fb864bf LogTimestamp=2022-04-26T06:21:18.989336Z ResponseDesc=Created Exiting. 2022-04-26T06:21:18.990Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:21:19.050Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=82888656-020a-444c-b456-3318caff7ea6 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:19.049536Z InvocationID=f545bced-421d-4965-b505-8d8d47eb1139 LogTimestamp=2022-04-26T06:21:19.049636Z Entering 2022-04-26T06:21:19.050Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=82888656-020a-444c-b456-3318caff7ea6 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:19.049536Z InvocationID=f545bced-421d-4965-b505-8d8d47eb1139 LogTimestamp=2022-04-26T06:21:19.049636Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:21:19.061Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=050e7625-bae0-4b9d-bd93-0b12c29f8210 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:21:19.061Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=050e7625-bae0-4b9d-bd93-0b12c29f8210 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4eb9b804 2022-04-26T06:21:19.062Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=050e7625-bae0-4b9d-bd93-0b12c29f8210 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:21:19.062Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=050e7625-bae0-4b9d-bd93-0b12c29f8210 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:21:19.199Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:19.051Z InvocationID=6ad17740-e724-4a75-9018-93beb5146e16 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:19.199Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:19.051Z InvocationID=6ad17740-e724-4a75-9018-93beb5146e16 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:19.199Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:19.051Z InvocationID=6ad17740-e724-4a75-9018-93beb5146e16 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:19.199Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:19.051Z InvocationID=6ad17740-e724-4a75-9018-93beb5146e16 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:19.213Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:19.051Z InvocationID=6ad17740-e724-4a75-9018-93beb5146e16 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Webapplication 2022-04-26T06:21:19.384Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "WebApplication" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "35c00487-8ca5-4f8c-84de-e1d3b3c4ee2f" INVARIANT_UUID = "cda51584-9097-4c91-a5e7-af6b7fe67949" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.WebApplication" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=35c00487-8ca5-4f8c-84de-e1d3b3c4ee2f EntryTimestamp=2022-04-26T06:21:19.051Z InvocationID=6ad17740-e724-4a75-9018-93beb5146e16 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 7625ddcb-b766-4b3d-ad56-f647b5f17a44 2022-04-26T06:21:20.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:21:20.379Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:21:20.389Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "WebApplication" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "35c00487-8ca5-4f8c-84de-e1d3b3c4ee2f" INVARIANT_UUID = "cda51584-9097-4c91-a5e7-af6b7fe67949" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "7625ddcb-b766-4b3d-ad56-f647b5f17a44.assettoscatemplate" CURR_ARTIFACT_UUID = "7625ddcb-b766-4b3d-ad56-f647b5f17a44.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Webapplication-template.yml,0,7625ddcb-b766-4b3d-ad56-f647b5f17a44.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=1337 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=fcdd1749-d30f-41df-a7a6-beaffc3ff658 EntryTimestamp=2022-04-26T06:21:19.051Z InvocationID=6ad17740-e724-4a75-9018-93beb5146e16 LogTimestamp=2022-04-26T06:21:20.388691Z ResponseDesc=Created Exiting. 2022-04-26T06:21:20.390Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:21:20.400Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=7465c2fc-6708-46ee-aeff-9c2baf8771b3 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:20.400399Z InvocationID=c715fd69-9a09-4501-970f-651fe03a2efd LogTimestamp=2022-04-26T06:21:20.400477Z Entering 2022-04-26T06:21:20.401Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=7465c2fc-6708-46ee-aeff-9c2baf8771b3 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:20.400399Z InvocationID=c715fd69-9a09-4501-970f-651fe03a2efd LogTimestamp=2022-04-26T06:21:20.400477Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:21:21.256Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:20.401Z InvocationID=f5857697-6613-47b8-b8ac-598cd3b4dc55 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:21.257Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:20.401Z InvocationID=f5857697-6613-47b8-b8ac-598cd3b4dc55 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:21.257Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:20.401Z InvocationID=f5857697-6613-47b8-b8ac-598cd3b4dc55 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:21.257Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:20.401Z InvocationID=f5857697-6613-47b8-b8ac-598cd3b4dc55 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:21.345Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:20.401Z InvocationID=f5857697-6613-47b8-b8ac-598cd3b4dc55 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Dbms 2022-04-26T06:21:21.867Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "DBMS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "ebf3cf73-dd12-4ab4-8d88-fa8ca47e24c6" INVARIANT_UUID = "1e9cca8c-a713-4c30-bf7e-85f89ed7761a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.DBMS" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ebf3cf73-dd12-4ab4-8d88-fa8ca47e24c6 EntryTimestamp=2022-04-26T06:21:20.401Z InvocationID=f5857697-6613-47b8-b8ac-598cd3b4dc55 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 64145e5f-4dc7-41ea-b708-6c0ccef6e0ab 2022-04-26T06:21:22.453Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2c65afc-e56c-4e2c-99d4-8062fe54c444 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:21:22.454Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2c65afc-e56c-4e2c-99d4-8062fe54c444 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@76c12b4d 2022-04-26T06:21:22.454Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2c65afc-e56c-4e2c-99d4-8062fe54c444 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:21:22.454Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2c65afc-e56c-4e2c-99d4-8062fe54c444 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:21:23.458Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "DBMS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "ebf3cf73-dd12-4ab4-8d88-fa8ca47e24c6" INVARIANT_UUID = "1e9cca8c-a713-4c30-bf7e-85f89ed7761a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "64145e5f-4dc7-41ea-b708-6c0ccef6e0ab.assettoscatemplate" CURR_ARTIFACT_UUID = "64145e5f-4dc7-41ea-b708-6c0ccef6e0ab.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Dbms-template.yml,0,64145e5f-4dc7-41ea-b708-6c0ccef6e0ab.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=3057 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b1cb811e-8ceb-45e2-a77f-3f53edd6bb00 EntryTimestamp=2022-04-26T06:21:20.401Z InvocationID=f5857697-6613-47b8-b8ac-598cd3b4dc55 LogTimestamp=2022-04-26T06:21:23.458257Z ResponseDesc=Created Exiting. 2022-04-26T06:21:23.459Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:21:23.558Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=83158e1c-f4f8-4710-8fcb-8e5ddb209539 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:23.557915Z InvocationID=43664858-f1f9-4c40-ac42-b29b2b7c9053 LogTimestamp=2022-04-26T06:21:23.558052Z Entering 2022-04-26T06:21:23.559Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=83158e1c-f4f8-4710-8fcb-8e5ddb209539 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:23.557915Z InvocationID=43664858-f1f9-4c40-ac42-b29b2b7c9053 LogTimestamp=2022-04-26T06:21:23.558052Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:21:24.062Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:23.559Z InvocationID=95287e36-bffa-49c6-b490-8bd71ac2ab94 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:24.062Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:23.559Z InvocationID=95287e36-bffa-49c6-b490-8bd71ac2ab94 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:24.062Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:23.559Z InvocationID=95287e36-bffa-49c6-b490-8bd71ac2ab94 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:24.063Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:23.559Z InvocationID=95287e36-bffa-49c6-b490-8bd71ac2ab94 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:24.155Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:23.559Z InvocationID=95287e36-bffa-49c6-b490-8bd71ac2ab94 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Database 2022-04-26T06:21:24.546Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Database" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "be343d06-0be7-4754-ae67-aac9a3823165" INVARIANT_UUID = "063ca83b-e988-4914-a02b-ea99e3e1df94" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.Database" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=be343d06-0be7-4754-ae67-aac9a3823165 EntryTimestamp=2022-04-26T06:21:23.559Z InvocationID=95287e36-bffa-49c6-b490-8bd71ac2ab94 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:21:24.547Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Database" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "be343d06-0be7-4754-ae67-aac9a3823165" INVARIANT_UUID = "063ca83b-e988-4914-a02b-ea99e3e1df94" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.Database" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=be343d06-0be7-4754-ae67-aac9a3823165 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:23.559Z InvocationID=95287e36-bffa-49c6-b490-8bd71ac2ab94 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:21:24.862Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Database" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "be343d06-0be7-4754-ae67-aac9a3823165" INVARIANT_UUID = "063ca83b-e988-4914-a02b-ea99e3e1df94" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.Database" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=be343d06-0be7-4754-ae67-aac9a3823165 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:23.559Z InvocationID=95287e36-bffa-49c6-b490-8bd71ac2ab94 ErrorCode=500 lock resource with id a6ad3f5c-d464-427b-929f-d87bdef178e0 2022-04-26T06:21:25.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae088416-5a5b-45cb-8c41-fc24f938e21e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:21:25.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae088416-5a5b-45cb-8c41-fc24f938e21e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6ce06c5d 2022-04-26T06:21:25.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae088416-5a5b-45cb-8c41-fc24f938e21e PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:21:25.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae088416-5a5b-45cb-8c41-fc24f938e21e PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:21:26.249Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Database" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "be343d06-0be7-4754-ae67-aac9a3823165" INVARIANT_UUID = "063ca83b-e988-4914-a02b-ea99e3e1df94" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a6ad3f5c-d464-427b-929f-d87bdef178e0.assettoscatemplate" CURR_ARTIFACT_UUID = "a6ad3f5c-d464-427b-929f-d87bdef178e0.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Database-template.yml,0,a6ad3f5c-d464-427b-929f-d87bdef178e0.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=990b322f-b841-41d1-a9d9-a9eb87bdd3ce TargetServiceName=null EntryTimestamp=2022-04-26T06:21:23.559Z InvocationID=95287e36-bffa-49c6-b490-8bd71ac2ab94 ErrorCode=500 Starting to create Csar for component Database 2022-04-26T06:21:26.492Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Database" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "be343d06-0be7-4754-ae67-aac9a3823165" INVARIANT_UUID = "063ca83b-e988-4914-a02b-ea99e3e1df94" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a6ad3f5c-d464-427b-929f-d87bdef178e0.assettoscatemplate" CURR_ARTIFACT_UUID = "a6ad3f5c-d464-427b-929f-d87bdef178e0.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Database-template.yml,0,a6ad3f5c-d464-427b-929f-d87bdef178e0.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=990b322f-b841-41d1-a9d9-a9eb87bdd3ce TargetServiceName=null EntryTimestamp=2022-04-26T06:21:23.559Z InvocationID=95287e36-bffa-49c6-b490-8bd71ac2ab94 ErrorCode=500 Ended create Csar for component Database 2022-04-26T06:21:26.492Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Database" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "be343d06-0be7-4754-ae67-aac9a3823165" INVARIANT_UUID = "063ca83b-e988-4914-a02b-ea99e3e1df94" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a6ad3f5c-d464-427b-929f-d87bdef178e0.assettoscatemplate" CURR_ARTIFACT_UUID = "a6ad3f5c-d464-427b-929f-d87bdef178e0.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Database-template.yml,0,a6ad3f5c-d464-427b-929f-d87bdef178e0.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=990b322f-b841-41d1-a9d9-a9eb87bdd3ce TargetServiceName=null EntryTimestamp=2022-04-26T06:21:23.559Z InvocationID=95287e36-bffa-49c6-b490-8bd71ac2ab94 ErrorCode=500 lock resource with id a6ad3f5c-d464-427b-929f-d87bdef178e0 2022-04-26T06:21:27.366Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Database" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "be343d06-0be7-4754-ae67-aac9a3823165" INVARIANT_UUID = "063ca83b-e988-4914-a02b-ea99e3e1df94" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a6ad3f5c-d464-427b-929f-d87bdef178e0.assettoscacsar" CURR_ARTIFACT_UUID = "a6ad3f5c-d464-427b-929f-d87bdef178e0.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Database-csar.csar,0,a6ad3f5c-d464-427b-929f-d87bdef178e0.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=3807 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f65fabe1-5d2b-4f8e-9c91-03c602d07364 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:23.559Z InvocationID=95287e36-bffa-49c6-b490-8bd71ac2ab94 ErrorCode=500 LogTimestamp=2022-04-26T06:21:27.366050Z ResponseDesc=Created Exiting. 2022-04-26T06:21:27.367Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:21:27.449Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=acaa38af-5c61-4d20-9228-e4b1c5b7f7d3 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:27.449031Z InvocationID=afa5f136-354c-465e-b62b-17dd9ec7257a LogTimestamp=2022-04-26T06:21:27.449124Z Entering 2022-04-26T06:21:27.449Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=acaa38af-5c61-4d20-9228-e4b1c5b7f7d3 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:27.449031Z InvocationID=afa5f136-354c-465e-b62b-17dd9ec7257a LogTimestamp=2022-04-26T06:21:27.449124Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:21:28.360Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:27.450Z InvocationID=20bb255b-0242-467d-8f7a-bcb1e061304c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:28.360Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:27.450Z InvocationID=20bb255b-0242-467d-8f7a-bcb1e061304c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:28.361Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:27.450Z InvocationID=20bb255b-0242-467d-8f7a-bcb1e061304c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:28.361Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:27.450Z InvocationID=20bb255b-0242-467d-8f7a-bcb1e061304c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:28.446Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:27.450Z InvocationID=20bb255b-0242-467d-8f7a-bcb1e061304c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Objectstorage 2022-04-26T06:21:28.454Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5350bf27-4a8d-4776-91aa-04d878dcca53 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:21:28.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5350bf27-4a8d-4776-91aa-04d878dcca53 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@70c1c9db 2022-04-26T06:21:28.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5350bf27-4a8d-4776-91aa-04d878dcca53 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:21:28.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5350bf27-4a8d-4776-91aa-04d878dcca53 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:21:28.750Z [qtp2085002312-20] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "083c835d-602a-46b1-8d0c-98f8e9c790d2" INVARIANT_UUID = "4c645c3b-50eb-4438-956f-e469a7d2edad" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.ObjectStorage" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=083c835d-602a-46b1-8d0c-98f8e9c790d2 EntryTimestamp=2022-04-26T06:21:27.450Z InvocationID=20bb255b-0242-467d-8f7a-bcb1e061304c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:21:28.752Z [qtp2085002312-20] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "083c835d-602a-46b1-8d0c-98f8e9c790d2" INVARIANT_UUID = "4c645c3b-50eb-4438-956f-e469a7d2edad" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.ObjectStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=083c835d-602a-46b1-8d0c-98f8e9c790d2 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:27.450Z InvocationID=20bb255b-0242-467d-8f7a-bcb1e061304c ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:21:29.071Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "083c835d-602a-46b1-8d0c-98f8e9c790d2" INVARIANT_UUID = "4c645c3b-50eb-4438-956f-e469a7d2edad" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.ObjectStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=083c835d-602a-46b1-8d0c-98f8e9c790d2 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:27.450Z InvocationID=20bb255b-0242-467d-8f7a-bcb1e061304c ErrorCode=500 lock resource with id bbd9f246-4e43-49e2-8b74-bb964a49eb3f 2022-04-26T06:21:31.445Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cd7b9433-f169-42ac-b625-4b0df3d695ea PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:21:31.446Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cd7b9433-f169-42ac-b625-4b0df3d695ea PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@733b265f 2022-04-26T06:21:31.446Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cd7b9433-f169-42ac-b625-4b0df3d695ea PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:21:31.446Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cd7b9433-f169-42ac-b625-4b0df3d695ea PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:21:32.644Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "083c835d-602a-46b1-8d0c-98f8e9c790d2" INVARIANT_UUID = "4c645c3b-50eb-4438-956f-e469a7d2edad" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "bbd9f246-4e43-49e2-8b74-bb964a49eb3f.assettoscatemplate" CURR_ARTIFACT_UUID = "bbd9f246-4e43-49e2-8b74-bb964a49eb3f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Objectstorage-template.yml,0,bbd9f246-4e43-49e2-8b74-bb964a49eb3f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d048ad01-4886-49fe-aee3-69b707a5808e TargetServiceName=null EntryTimestamp=2022-04-26T06:21:27.450Z InvocationID=20bb255b-0242-467d-8f7a-bcb1e061304c ErrorCode=500 Starting to create Csar for component ObjectStorage 2022-04-26T06:21:32.905Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "083c835d-602a-46b1-8d0c-98f8e9c790d2" INVARIANT_UUID = "4c645c3b-50eb-4438-956f-e469a7d2edad" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "bbd9f246-4e43-49e2-8b74-bb964a49eb3f.assettoscatemplate" CURR_ARTIFACT_UUID = "bbd9f246-4e43-49e2-8b74-bb964a49eb3f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Objectstorage-template.yml,0,bbd9f246-4e43-49e2-8b74-bb964a49eb3f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d048ad01-4886-49fe-aee3-69b707a5808e TargetServiceName=null EntryTimestamp=2022-04-26T06:21:27.450Z InvocationID=20bb255b-0242-467d-8f7a-bcb1e061304c ErrorCode=500 Ended create Csar for component ObjectStorage 2022-04-26T06:21:32.906Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "083c835d-602a-46b1-8d0c-98f8e9c790d2" INVARIANT_UUID = "4c645c3b-50eb-4438-956f-e469a7d2edad" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "bbd9f246-4e43-49e2-8b74-bb964a49eb3f.assettoscatemplate" CURR_ARTIFACT_UUID = "bbd9f246-4e43-49e2-8b74-bb964a49eb3f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Objectstorage-template.yml,0,bbd9f246-4e43-49e2-8b74-bb964a49eb3f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d048ad01-4886-49fe-aee3-69b707a5808e TargetServiceName=null EntryTimestamp=2022-04-26T06:21:27.450Z InvocationID=20bb255b-0242-467d-8f7a-bcb1e061304c ErrorCode=500 lock resource with id bbd9f246-4e43-49e2-8b74-bb964a49eb3f 2022-04-26T06:21:34.280Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "083c835d-602a-46b1-8d0c-98f8e9c790d2" INVARIANT_UUID = "4c645c3b-50eb-4438-956f-e469a7d2edad" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "bbd9f246-4e43-49e2-8b74-bb964a49eb3f.assettoscacsar" CURR_ARTIFACT_UUID = "bbd9f246-4e43-49e2-8b74-bb964a49eb3f.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Objectstorage-csar.csar,0,bbd9f246-4e43-49e2-8b74-bb964a49eb3f.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=6829 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b7fc13e7-665b-4967-b752-3dee11ffe8e1 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:27.450Z InvocationID=20bb255b-0242-467d-8f7a-bcb1e061304c ErrorCode=500 LogTimestamp=2022-04-26T06:21:34.279458Z ResponseDesc=Created Exiting. 2022-04-26T06:21:34.281Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:21:34.454Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18395f8c-16d8-401c-a9e6-6c63d69cb19f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:21:34.454Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18395f8c-16d8-401c-a9e6-6c63d69cb19f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@cdfe036 2022-04-26T06:21:34.454Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18395f8c-16d8-401c-a9e6-6c63d69cb19f PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:21:34.454Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18395f8c-16d8-401c-a9e6-6c63d69cb19f PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:21:34.458Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=3767af57-2931-4dde-8e3e-c9e47e04695f InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:34.457630Z InvocationID=c4a73015-bf28-4cf5-ac90-9abfd2a1c206 LogTimestamp=2022-04-26T06:21:34.457731Z Entering 2022-04-26T06:21:34.459Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=3767af57-2931-4dde-8e3e-c9e47e04695f InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:34.457630Z InvocationID=c4a73015-bf28-4cf5-ac90-9abfd2a1c206 LogTimestamp=2022-04-26T06:21:34.457731Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:21:35.163Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:34.460Z InvocationID=6e627164-798d-4ab7-a69c-8619199ded81 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:35.164Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:34.460Z InvocationID=6e627164-798d-4ab7-a69c-8619199ded81 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:35.164Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:34.460Z InvocationID=6e627164-798d-4ab7-a69c-8619199ded81 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:35.164Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:34.460Z InvocationID=6e627164-798d-4ab7-a69c-8619199ded81 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:35.237Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:34.460Z InvocationID=6e627164-798d-4ab7-a69c-8619199ded81 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Blockstorage 2022-04-26T06:21:35.368Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:21:35.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:21:35.655Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "BlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "f4486db3-4265-4c71-9a76-81cf4aafa920" INVARIANT_UUID = "d09f4238-1fc4-4b84-97e1-e6531dfb637b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.BlockStorage" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f4486db3-4265-4c71-9a76-81cf4aafa920 EntryTimestamp=2022-04-26T06:21:34.460Z InvocationID=6e627164-798d-4ab7-a69c-8619199ded81 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:21:35.656Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "BlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "f4486db3-4265-4c71-9a76-81cf4aafa920" INVARIANT_UUID = "d09f4238-1fc4-4b84-97e1-e6531dfb637b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.BlockStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f4486db3-4265-4c71-9a76-81cf4aafa920 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:34.460Z InvocationID=6e627164-798d-4ab7-a69c-8619199ded81 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:21:35.961Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "BlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "f4486db3-4265-4c71-9a76-81cf4aafa920" INVARIANT_UUID = "d09f4238-1fc4-4b84-97e1-e6531dfb637b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.BlockStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f4486db3-4265-4c71-9a76-81cf4aafa920 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:34.460Z InvocationID=6e627164-798d-4ab7-a69c-8619199ded81 ErrorCode=500 lock resource with id 1e31269f-e244-4a22-ac72-5d92feae4ab1 2022-04-26T06:21:37.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a15759b2-cd85-4c4c-98ab-a85726941ca2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:21:37.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a15759b2-cd85-4c4c-98ab-a85726941ca2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@63b2f774 2022-04-26T06:21:37.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a15759b2-cd85-4c4c-98ab-a85726941ca2 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:21:37.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a15759b2-cd85-4c4c-98ab-a85726941ca2 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:21:37.630Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "BlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "f4486db3-4265-4c71-9a76-81cf4aafa920" INVARIANT_UUID = "d09f4238-1fc4-4b84-97e1-e6531dfb637b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1e31269f-e244-4a22-ac72-5d92feae4ab1.assettoscatemplate" CURR_ARTIFACT_UUID = "1e31269f-e244-4a22-ac72-5d92feae4ab1.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Blockstorage-template.yml,0,1e31269f-e244-4a22-ac72-5d92feae4ab1.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a97453d2-4576-4a25-bcff-3b22e8f45982 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:34.460Z InvocationID=6e627164-798d-4ab7-a69c-8619199ded81 ErrorCode=500 Starting to create Csar for component BlockStorage 2022-04-26T06:21:37.897Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "BlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "f4486db3-4265-4c71-9a76-81cf4aafa920" INVARIANT_UUID = "d09f4238-1fc4-4b84-97e1-e6531dfb637b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1e31269f-e244-4a22-ac72-5d92feae4ab1.assettoscatemplate" CURR_ARTIFACT_UUID = "1e31269f-e244-4a22-ac72-5d92feae4ab1.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Blockstorage-template.yml,0,1e31269f-e244-4a22-ac72-5d92feae4ab1.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a97453d2-4576-4a25-bcff-3b22e8f45982 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:34.460Z InvocationID=6e627164-798d-4ab7-a69c-8619199ded81 ErrorCode=500 Ended create Csar for component BlockStorage 2022-04-26T06:21:37.898Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "BlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "f4486db3-4265-4c71-9a76-81cf4aafa920" INVARIANT_UUID = "d09f4238-1fc4-4b84-97e1-e6531dfb637b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1e31269f-e244-4a22-ac72-5d92feae4ab1.assettoscatemplate" CURR_ARTIFACT_UUID = "1e31269f-e244-4a22-ac72-5d92feae4ab1.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Blockstorage-template.yml,0,1e31269f-e244-4a22-ac72-5d92feae4ab1.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a97453d2-4576-4a25-bcff-3b22e8f45982 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:34.460Z InvocationID=6e627164-798d-4ab7-a69c-8619199ded81 ErrorCode=500 lock resource with id 1e31269f-e244-4a22-ac72-5d92feae4ab1 2022-04-26T06:21:38.876Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "BlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "f4486db3-4265-4c71-9a76-81cf4aafa920" INVARIANT_UUID = "d09f4238-1fc4-4b84-97e1-e6531dfb637b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1e31269f-e244-4a22-ac72-5d92feae4ab1.assettoscacsar" CURR_ARTIFACT_UUID = "1e31269f-e244-4a22-ac72-5d92feae4ab1.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Blockstorage-csar.csar,0,1e31269f-e244-4a22-ac72-5d92feae4ab1.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=4415 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7470378f-035b-4a72-bc22-2c33b2e28644 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:34.460Z InvocationID=6e627164-798d-4ab7-a69c-8619199ded81 ErrorCode=500 LogTimestamp=2022-04-26T06:21:38.875876Z ResponseDesc=Created Exiting. 2022-04-26T06:21:38.877Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:21:38.941Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=56b429c7-c0c4-4181-86d6-7af6fb82145a InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:38.941385Z InvocationID=bb877a05-f9f6-4f73-a66f-d60b8e859aa5 LogTimestamp=2022-04-26T06:21:38.941477Z Entering 2022-04-26T06:21:38.946Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=56b429c7-c0c4-4181-86d6-7af6fb82145a InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:38.941385Z InvocationID=bb877a05-f9f6-4f73-a66f-d60b8e859aa5 LogTimestamp=2022-04-26T06:21:38.941477Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:21:39.272Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:38.946Z InvocationID=a8af90cd-5bbb-4207-85e2-9760c17059c6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:39.272Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:38.946Z InvocationID=a8af90cd-5bbb-4207-85e2-9760c17059c6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:39.272Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:38.946Z InvocationID=a8af90cd-5bbb-4207-85e2-9760c17059c6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:39.273Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:38.946Z InvocationID=a8af90cd-5bbb-4207-85e2-9760c17059c6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:39.340Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:38.946Z InvocationID=a8af90cd-5bbb-4207-85e2-9760c17059c6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Runtime 2022-04-26T06:21:39.500Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Runtime" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1533f8b6-f252-4c36-8781-fa172f8198fc" INVARIANT_UUID = "8aa4e00a-991f-4537-b43d-6a9a3395db12" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.Container.Runtime" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1533f8b6-f252-4c36-8781-fa172f8198fc EntryTimestamp=2022-04-26T06:21:38.946Z InvocationID=a8af90cd-5bbb-4207-85e2-9760c17059c6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 78f0083e-0976-4a61-9af2-921683571e81 2022-04-26T06:21:39.791Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe85ff6f-1e2b-47a1-871c-dad22679e8b2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:21:39.792Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe85ff6f-1e2b-47a1-871c-dad22679e8b2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1d4365c8 2022-04-26T06:21:39.792Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe85ff6f-1e2b-47a1-871c-dad22679e8b2 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:21:39.792Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe85ff6f-1e2b-47a1-871c-dad22679e8b2 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:21:40.189Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Runtime" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1533f8b6-f252-4c36-8781-fa172f8198fc" INVARIANT_UUID = "8aa4e00a-991f-4537-b43d-6a9a3395db12" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "78f0083e-0976-4a61-9af2-921683571e81.assettoscatemplate" CURR_ARTIFACT_UUID = "78f0083e-0976-4a61-9af2-921683571e81.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Runtime-template.yml,0,78f0083e-0976-4a61-9af2-921683571e81.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=1242 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2892934d-5907-45bc-bd3a-6166bf18fdee EntryTimestamp=2022-04-26T06:21:38.946Z InvocationID=a8af90cd-5bbb-4207-85e2-9760c17059c6 LogTimestamp=2022-04-26T06:21:40.188801Z ResponseDesc=Created Exiting. 2022-04-26T06:21:40.189Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:21:40.202Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=d3ec3757-9a6a-4358-9d5d-30f9a6301cc7 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:40.202361Z InvocationID=00910d03-e14e-4fad-819d-7aea01d13e83 LogTimestamp=2022-04-26T06:21:40.202490Z Entering 2022-04-26T06:21:40.203Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=d3ec3757-9a6a-4358-9d5d-30f9a6301cc7 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:40.202361Z InvocationID=00910d03-e14e-4fad-819d-7aea01d13e83 LogTimestamp=2022-04-26T06:21:40.202490Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:21:40.321Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:40.203Z InvocationID=184f386c-d12f-4fce-ae9c-e5ad057fa996 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:40.321Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:40.203Z InvocationID=184f386c-d12f-4fce-ae9c-e5ad057fa996 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:40.321Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:40.203Z InvocationID=184f386c-d12f-4fce-ae9c-e5ad057fa996 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:40.322Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:40.203Z InvocationID=184f386c-d12f-4fce-ae9c-e5ad057fa996 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:40.329Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:40.203Z InvocationID=184f386c-d12f-4fce-ae9c-e5ad057fa996 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Application 2022-04-26T06:21:40.421Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Application" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "3c623cc9-98f6-4a30-a908-a9a9e329ba4c" INVARIANT_UUID = "a9a719d4-f86a-427d-8997-9d12398e17ab" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.Container.Application" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3c623cc9-98f6-4a30-a908-a9a9e329ba4c EntryTimestamp=2022-04-26T06:21:40.203Z InvocationID=184f386c-d12f-4fce-ae9c-e5ad057fa996 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id ac0154da-ad59-4059-b2fa-839b8f977643 2022-04-26T06:21:41.059Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Application" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "3c623cc9-98f6-4a30-a908-a9a9e329ba4c" INVARIANT_UUID = "a9a719d4-f86a-427d-8997-9d12398e17ab" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ac0154da-ad59-4059-b2fa-839b8f977643.assettoscatemplate" CURR_ARTIFACT_UUID = "ac0154da-ad59-4059-b2fa-839b8f977643.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Application-template.yml,0,ac0154da-ad59-4059-b2fa-839b8f977643.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=856 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6e57c5cf-6659-4920-a351-45023ee4716d EntryTimestamp=2022-04-26T06:21:40.203Z InvocationID=184f386c-d12f-4fce-ae9c-e5ad057fa996 LogTimestamp=2022-04-26T06:21:41.059099Z ResponseDesc=Created Exiting. 2022-04-26T06:21:41.060Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:21:41.070Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=8bdcdcbc-2f4c-4441-8878-2201c3fa890b InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:41.070124Z InvocationID=6276a9b4-2dc1-4ca0-8a3a-3dde54848968 LogTimestamp=2022-04-26T06:21:41.070268Z Entering 2022-04-26T06:21:41.070Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=8bdcdcbc-2f4c-4441-8878-2201c3fa890b InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:41.070124Z InvocationID=6276a9b4-2dc1-4ca0-8a3a-3dde54848968 LogTimestamp=2022-04-26T06:21:41.070268Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:21:41.172Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:41.071Z InvocationID=4290926a-167e-46a0-8b17-4fed2b760da2 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:41.172Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:41.071Z InvocationID=4290926a-167e-46a0-8b17-4fed2b760da2 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:41.172Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:41.071Z InvocationID=4290926a-167e-46a0-8b17-4fed2b760da2 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:41.172Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:41.071Z InvocationID=4290926a-167e-46a0-8b17-4fed2b760da2 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:41.202Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:41.071Z InvocationID=4290926a-167e-46a0-8b17-4fed2b760da2 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Loadbalancer 2022-04-26T06:21:41.260Z [qtp2085002312-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "LoadBalancer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "56458a52-c0c1-4382-9d74-3f63f1ef32f2" INVARIANT_UUID = "684c6727-cb39-49e0-8d4a-c29f27441ce7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.LoadBalancer" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=56458a52-c0c1-4382-9d74-3f63f1ef32f2 EntryTimestamp=2022-04-26T06:21:41.071Z InvocationID=4290926a-167e-46a0-8b17-4fed2b760da2 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:21:41.260Z [qtp2085002312-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "LoadBalancer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "56458a52-c0c1-4382-9d74-3f63f1ef32f2" INVARIANT_UUID = "684c6727-cb39-49e0-8d4a-c29f27441ce7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.LoadBalancer" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=56458a52-c0c1-4382-9d74-3f63f1ef32f2 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:41.071Z InvocationID=4290926a-167e-46a0-8b17-4fed2b760da2 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:21:41.347Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "LoadBalancer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "56458a52-c0c1-4382-9d74-3f63f1ef32f2" INVARIANT_UUID = "684c6727-cb39-49e0-8d4a-c29f27441ce7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.LoadBalancer" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=56458a52-c0c1-4382-9d74-3f63f1ef32f2 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:41.071Z InvocationID=4290926a-167e-46a0-8b17-4fed2b760da2 ErrorCode=500 lock resource with id 911c1040-218f-4b3a-ac65-22761977e0ba 2022-04-26T06:21:42.049Z [qtp2085002312-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "LoadBalancer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "56458a52-c0c1-4382-9d74-3f63f1ef32f2" INVARIANT_UUID = "684c6727-cb39-49e0-8d4a-c29f27441ce7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "911c1040-218f-4b3a-ac65-22761977e0ba.assettoscatemplate" CURR_ARTIFACT_UUID = "911c1040-218f-4b3a-ac65-22761977e0ba.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Loadbalancer-template.yml,0,911c1040-218f-4b3a-ac65-22761977e0ba.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=caf4bea1-7d6c-4fc7-a8a0-f1776d557f1f TargetServiceName=null EntryTimestamp=2022-04-26T06:21:41.071Z InvocationID=4290926a-167e-46a0-8b17-4fed2b760da2 ErrorCode=500 Starting to create Csar for component LoadBalancer 2022-04-26T06:21:42.291Z [qtp2085002312-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "LoadBalancer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "56458a52-c0c1-4382-9d74-3f63f1ef32f2" INVARIANT_UUID = "684c6727-cb39-49e0-8d4a-c29f27441ce7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "911c1040-218f-4b3a-ac65-22761977e0ba.assettoscatemplate" CURR_ARTIFACT_UUID = "911c1040-218f-4b3a-ac65-22761977e0ba.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Loadbalancer-template.yml,0,911c1040-218f-4b3a-ac65-22761977e0ba.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=caf4bea1-7d6c-4fc7-a8a0-f1776d557f1f TargetServiceName=null EntryTimestamp=2022-04-26T06:21:41.071Z InvocationID=4290926a-167e-46a0-8b17-4fed2b760da2 ErrorCode=500 Ended create Csar for component LoadBalancer 2022-04-26T06:21:42.292Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "LoadBalancer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "56458a52-c0c1-4382-9d74-3f63f1ef32f2" INVARIANT_UUID = "684c6727-cb39-49e0-8d4a-c29f27441ce7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "911c1040-218f-4b3a-ac65-22761977e0ba.assettoscatemplate" CURR_ARTIFACT_UUID = "911c1040-218f-4b3a-ac65-22761977e0ba.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Loadbalancer-template.yml,0,911c1040-218f-4b3a-ac65-22761977e0ba.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=caf4bea1-7d6c-4fc7-a8a0-f1776d557f1f TargetServiceName=null EntryTimestamp=2022-04-26T06:21:41.071Z InvocationID=4290926a-167e-46a0-8b17-4fed2b760da2 ErrorCode=500 lock resource with id 911c1040-218f-4b3a-ac65-22761977e0ba 2022-04-26T06:21:42.790Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4ab7b31-2651-4c18-9594-9c528aa6a05a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:21:42.791Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4ab7b31-2651-4c18-9594-9c528aa6a05a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3760b1e1 2022-04-26T06:21:42.791Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4ab7b31-2651-4c18-9594-9c528aa6a05a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:21:42.791Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4ab7b31-2651-4c18-9594-9c528aa6a05a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:21:42.886Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "LoadBalancer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "56458a52-c0c1-4382-9d74-3f63f1ef32f2" INVARIANT_UUID = "684c6727-cb39-49e0-8d4a-c29f27441ce7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "911c1040-218f-4b3a-ac65-22761977e0ba.assettoscacsar" CURR_ARTIFACT_UUID = "911c1040-218f-4b3a-ac65-22761977e0ba.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Loadbalancer-csar.csar,0,911c1040-218f-4b3a-ac65-22761977e0ba.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1814 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c0dfb5c7-eb5f-457c-952e-e5f525e2831c TargetServiceName=null EntryTimestamp=2022-04-26T06:21:41.071Z InvocationID=4290926a-167e-46a0-8b17-4fed2b760da2 ErrorCode=500 LogTimestamp=2022-04-26T06:21:42.885797Z ResponseDesc=Created Exiting. 2022-04-26T06:21:42.886Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:21:42.902Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=e7cb8d47-908a-434c-907c-4479eab29fe4 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:42.902442Z InvocationID=29a3be2c-80d2-4ec2-bb35-564c31eb63d4 LogTimestamp=2022-04-26T06:21:42.902531Z Entering 2022-04-26T06:21:42.903Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=e7cb8d47-908a-434c-907c-4479eab29fe4 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:42.902442Z InvocationID=29a3be2c-80d2-4ec2-bb35-564c31eb63d4 LogTimestamp=2022-04-26T06:21:42.902531Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:21:43.020Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:42.903Z InvocationID=88d657c5-8513-4558-8f3d-20c862b88d55 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:43.020Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:42.903Z InvocationID=88d657c5-8513-4558-8f3d-20c862b88d55 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:43.021Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:42.903Z InvocationID=88d657c5-8513-4558-8f3d-20c862b88d55 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:43.021Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:42.903Z InvocationID=88d657c5-8513-4558-8f3d-20c862b88d55 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:43.033Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:42.903Z InvocationID=88d657c5-8513-4558-8f3d-20c862b88d55 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Port 2022-04-26T06:21:43.099Z [qtp2085002312-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Port" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "7edfdab5-1282-48b2-a998-d46ede6da14a" INVARIANT_UUID = "9c6224e2-25ed-4b85-8520-a203f490e861" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.network.Port" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7edfdab5-1282-48b2-a998-d46ede6da14a EntryTimestamp=2022-04-26T06:21:42.903Z InvocationID=88d657c5-8513-4558-8f3d-20c862b88d55 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:21:43.101Z [qtp2085002312-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Port" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "7edfdab5-1282-48b2-a998-d46ede6da14a" INVARIANT_UUID = "9c6224e2-25ed-4b85-8520-a203f490e861" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.network.Port" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7edfdab5-1282-48b2-a998-d46ede6da14a TargetServiceName=null EntryTimestamp=2022-04-26T06:21:42.903Z InvocationID=88d657c5-8513-4558-8f3d-20c862b88d55 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:21:43.215Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Port" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "7edfdab5-1282-48b2-a998-d46ede6da14a" INVARIANT_UUID = "9c6224e2-25ed-4b85-8520-a203f490e861" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.network.Port" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7edfdab5-1282-48b2-a998-d46ede6da14a TargetServiceName=null EntryTimestamp=2022-04-26T06:21:42.903Z InvocationID=88d657c5-8513-4558-8f3d-20c862b88d55 ErrorCode=500 lock resource with id ae2c48dd-ba38-46d1-ab56-490e43669b4f 2022-04-26T06:21:43.962Z [qtp2085002312-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Port" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "7edfdab5-1282-48b2-a998-d46ede6da14a" INVARIANT_UUID = "9c6224e2-25ed-4b85-8520-a203f490e861" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ae2c48dd-ba38-46d1-ab56-490e43669b4f.assettoscatemplate" CURR_ARTIFACT_UUID = "ae2c48dd-ba38-46d1-ab56-490e43669b4f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Port-template.yml,0,ae2c48dd-ba38-46d1-ab56-490e43669b4f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2d6d96ac-ea1e-475f-9fc5-8c75e494128b TargetServiceName=null EntryTimestamp=2022-04-26T06:21:42.903Z InvocationID=88d657c5-8513-4558-8f3d-20c862b88d55 ErrorCode=500 Starting to create Csar for component Port 2022-04-26T06:21:44.211Z [qtp2085002312-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Port" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "7edfdab5-1282-48b2-a998-d46ede6da14a" INVARIANT_UUID = "9c6224e2-25ed-4b85-8520-a203f490e861" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ae2c48dd-ba38-46d1-ab56-490e43669b4f.assettoscatemplate" CURR_ARTIFACT_UUID = "ae2c48dd-ba38-46d1-ab56-490e43669b4f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Port-template.yml,0,ae2c48dd-ba38-46d1-ab56-490e43669b4f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2d6d96ac-ea1e-475f-9fc5-8c75e494128b TargetServiceName=null EntryTimestamp=2022-04-26T06:21:42.903Z InvocationID=88d657c5-8513-4558-8f3d-20c862b88d55 ErrorCode=500 Ended create Csar for component Port 2022-04-26T06:21:44.211Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Port" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "7edfdab5-1282-48b2-a998-d46ede6da14a" INVARIANT_UUID = "9c6224e2-25ed-4b85-8520-a203f490e861" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ae2c48dd-ba38-46d1-ab56-490e43669b4f.assettoscatemplate" CURR_ARTIFACT_UUID = "ae2c48dd-ba38-46d1-ab56-490e43669b4f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Port-template.yml,0,ae2c48dd-ba38-46d1-ab56-490e43669b4f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2d6d96ac-ea1e-475f-9fc5-8c75e494128b TargetServiceName=null EntryTimestamp=2022-04-26T06:21:42.903Z InvocationID=88d657c5-8513-4558-8f3d-20c862b88d55 ErrorCode=500 lock resource with id ae2c48dd-ba38-46d1-ab56-490e43669b4f 2022-04-26T06:21:44.797Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Port" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "7edfdab5-1282-48b2-a998-d46ede6da14a" INVARIANT_UUID = "9c6224e2-25ed-4b85-8520-a203f490e861" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ae2c48dd-ba38-46d1-ab56-490e43669b4f.assettoscacsar" CURR_ARTIFACT_UUID = "ae2c48dd-ba38-46d1-ab56-490e43669b4f.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Port-csar.csar,0,ae2c48dd-ba38-46d1-ab56-490e43669b4f.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1894 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=89d38205-c83c-4c47-a19c-4197dbb41ed5 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:42.903Z InvocationID=88d657c5-8513-4558-8f3d-20c862b88d55 ErrorCode=500 LogTimestamp=2022-04-26T06:21:44.797345Z ResponseDesc=Created Exiting. 2022-04-26T06:21:44.798Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:21:44.846Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a841d2fb-c74d-4e5a-bcd0-5a1a8e48178d InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:44.845735Z InvocationID=253f6ae3-8645-4a08-abef-a68db461309a LogTimestamp=2022-04-26T06:21:44.845832Z Entering 2022-04-26T06:21:44.846Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a841d2fb-c74d-4e5a-bcd0-5a1a8e48178d InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:44.845735Z InvocationID=253f6ae3-8645-4a08-abef-a68db461309a LogTimestamp=2022-04-26T06:21:44.845832Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:21:45.012Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:44.846Z InvocationID=1b1330b8-fdfc-4218-a26f-bb934720a4d6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:45.012Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:44.846Z InvocationID=1b1330b8-fdfc-4218-a26f-bb934720a4d6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:45.012Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:44.846Z InvocationID=1b1330b8-fdfc-4218-a26f-bb934720a4d6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:45.012Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:44.846Z InvocationID=1b1330b8-fdfc-4218-a26f-bb934720a4d6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:45.017Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:44.846Z InvocationID=1b1330b8-fdfc-4218-a26f-bb934720a4d6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Network 2022-04-26T06:21:45.066Z [qtp2085002312-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Network" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "00f03b25-09f8-447b-9a84-ae1078b93fb8" INVARIANT_UUID = "b466106c-de65-40da-807d-ae38bc3fcea2" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.network.Network" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=00f03b25-09f8-447b-9a84-ae1078b93fb8 EntryTimestamp=2022-04-26T06:21:44.846Z InvocationID=1b1330b8-fdfc-4218-a26f-bb934720a4d6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:21:45.067Z [qtp2085002312-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Network" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "00f03b25-09f8-447b-9a84-ae1078b93fb8" INVARIANT_UUID = "b466106c-de65-40da-807d-ae38bc3fcea2" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.network.Network" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=00f03b25-09f8-447b-9a84-ae1078b93fb8 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:44.846Z InvocationID=1b1330b8-fdfc-4218-a26f-bb934720a4d6 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:21:45.110Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Network" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "00f03b25-09f8-447b-9a84-ae1078b93fb8" INVARIANT_UUID = "b466106c-de65-40da-807d-ae38bc3fcea2" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.network.Network" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=00f03b25-09f8-447b-9a84-ae1078b93fb8 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:44.846Z InvocationID=1b1330b8-fdfc-4218-a26f-bb934720a4d6 ErrorCode=500 lock resource with id be1fdd7a-ee02-417a-a44c-38fb6e4c6f01 2022-04-26T06:21:45.864Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Network" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "00f03b25-09f8-447b-9a84-ae1078b93fb8" INVARIANT_UUID = "b466106c-de65-40da-807d-ae38bc3fcea2" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "be1fdd7a-ee02-417a-a44c-38fb6e4c6f01.assettoscatemplate" CURR_ARTIFACT_UUID = "be1fdd7a-ee02-417a-a44c-38fb6e4c6f01.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Network-template.yml,0,be1fdd7a-ee02-417a-a44c-38fb6e4c6f01.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e4fa2446-8a9e-48ab-9440-f036af8a04c7 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:44.846Z InvocationID=1b1330b8-fdfc-4218-a26f-bb934720a4d6 ErrorCode=500 Starting to create Csar for component Network 2022-04-26T06:21:45.945Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de2aee17-3145-47f5-a79a-6f8e4b507b39 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:21:45.946Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de2aee17-3145-47f5-a79a-6f8e4b507b39 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@44cd396b 2022-04-26T06:21:45.946Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de2aee17-3145-47f5-a79a-6f8e4b507b39 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:21:45.946Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de2aee17-3145-47f5-a79a-6f8e4b507b39 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:21:46.142Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Network" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "00f03b25-09f8-447b-9a84-ae1078b93fb8" INVARIANT_UUID = "b466106c-de65-40da-807d-ae38bc3fcea2" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "be1fdd7a-ee02-417a-a44c-38fb6e4c6f01.assettoscatemplate" CURR_ARTIFACT_UUID = "be1fdd7a-ee02-417a-a44c-38fb6e4c6f01.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Network-template.yml,0,be1fdd7a-ee02-417a-a44c-38fb6e4c6f01.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e4fa2446-8a9e-48ab-9440-f036af8a04c7 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:44.846Z InvocationID=1b1330b8-fdfc-4218-a26f-bb934720a4d6 ErrorCode=500 Ended create Csar for component Network 2022-04-26T06:21:46.143Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Network" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "00f03b25-09f8-447b-9a84-ae1078b93fb8" INVARIANT_UUID = "b466106c-de65-40da-807d-ae38bc3fcea2" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "be1fdd7a-ee02-417a-a44c-38fb6e4c6f01.assettoscatemplate" CURR_ARTIFACT_UUID = "be1fdd7a-ee02-417a-a44c-38fb6e4c6f01.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Network-template.yml,0,be1fdd7a-ee02-417a-a44c-38fb6e4c6f01.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e4fa2446-8a9e-48ab-9440-f036af8a04c7 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:44.846Z InvocationID=1b1330b8-fdfc-4218-a26f-bb934720a4d6 ErrorCode=500 lock resource with id be1fdd7a-ee02-417a-a44c-38fb6e4c6f01 2022-04-26T06:21:46.703Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Network" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "00f03b25-09f8-447b-9a84-ae1078b93fb8" INVARIANT_UUID = "b466106c-de65-40da-807d-ae38bc3fcea2" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "be1fdd7a-ee02-417a-a44c-38fb6e4c6f01.assettoscacsar" CURR_ARTIFACT_UUID = "be1fdd7a-ee02-417a-a44c-38fb6e4c6f01.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Network-csar.csar,0,be1fdd7a-ee02-417a-a44c-38fb6e4c6f01.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1857 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b8b94fab-0b73-479a-909b-5083a32bdbee TargetServiceName=null EntryTimestamp=2022-04-26T06:21:44.846Z InvocationID=1b1330b8-fdfc-4218-a26f-bb934720a4d6 ErrorCode=500 LogTimestamp=2022-04-26T06:21:46.703032Z ResponseDesc=Created Exiting. 2022-04-26T06:21:46.704Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:21:46.720Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6631ab96-b495-4be0-b2eb-b87da73c329d InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:46.720383Z InvocationID=dd56a3d5-0688-4051-b631-704ef3981d6c LogTimestamp=2022-04-26T06:21:46.720471Z Entering 2022-04-26T06:21:46.721Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6631ab96-b495-4be0-b2eb-b87da73c329d InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:46.720383Z InvocationID=dd56a3d5-0688-4051-b631-704ef3981d6c LogTimestamp=2022-04-26T06:21:46.720471Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:21:46.859Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:46.721Z InvocationID=2094b59c-26eb-4520-82f4-015ff86e62a5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:46.859Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:46.721Z InvocationID=2094b59c-26eb-4520-82f4-015ff86e62a5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:46.860Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:46.721Z InvocationID=2094b59c-26eb-4520-82f4-015ff86e62a5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:46.860Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:46.721Z InvocationID=2094b59c-26eb-4520-82f4-015ff86e62a5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:46.868Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:46.721Z InvocationID=2094b59c-26eb-4520-82f4-015ff86e62a5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Globalnetwork 2022-04-26T06:21:46.955Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "GlobalNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "6bc28c6e-fcdb-4169-92fe-927c965d5196" INVARIANT_UUID = "d09cc390-d0ad-43cb-a87c-be2803f352f8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.nodes.network.Network" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6bc28c6e-fcdb-4169-92fe-927c965d5196 EntryTimestamp=2022-04-26T06:21:46.721Z InvocationID=2094b59c-26eb-4520-82f4-015ff86e62a5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id d5771257-9019-4008-ada9-099e257feebb 2022-04-26T06:21:47.594Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "GlobalNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "6bc28c6e-fcdb-4169-92fe-927c965d5196" INVARIANT_UUID = "d09cc390-d0ad-43cb-a87c-be2803f352f8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "d5771257-9019-4008-ada9-099e257feebb.assettoscatemplate" CURR_ARTIFACT_UUID = "d5771257-9019-4008-ada9-099e257feebb.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Globalnetwork-template.yml,0,d5771257-9019-4008-ada9-099e257feebb.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=873 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ff90767a-b033-4bba-a354-10472f3543fe EntryTimestamp=2022-04-26T06:21:46.721Z InvocationID=2094b59c-26eb-4520-82f4-015ff86e62a5 LogTimestamp=2022-04-26T06:21:47.594651Z ResponseDesc=Created Exiting. 2022-04-26T06:21:47.595Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:21:47.632Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=24e1bcfc-fb59-4673-8c4d-cbf1ee4fdfd5 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:47.631766Z InvocationID=d8ec8976-79f7-4cd0-b12d-629ca7d881ac LogTimestamp=2022-04-26T06:21:47.631907Z Entering 2022-04-26T06:21:47.632Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=24e1bcfc-fb59-4673-8c4d-cbf1ee4fdfd5 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:47.631766Z InvocationID=d8ec8976-79f7-4cd0-b12d-629ca7d881ac LogTimestamp=2022-04-26T06:21:47.631907Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:21:47.906Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:47.633Z InvocationID=04e65402-0b88-4f5c-afc8-6153c61a5be1 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:47.906Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:47.633Z InvocationID=04e65402-0b88-4f5c-afc8-6153c61a5be1 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:47.906Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:47.633Z InvocationID=04e65402-0b88-4f5c-afc8-6153c61a5be1 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:47.906Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:47.633Z InvocationID=04e65402-0b88-4f5c-afc8-6153c61a5be1 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:47.928Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:47.633Z InvocationID=04e65402-0b88-4f5c-afc8-6153c61a5be1 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Globalport 2022-04-26T06:21:48.028Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "GlobalPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "9fa9ab72-affd-45ba-939c-f5b19c965310" INVARIANT_UUID = "087707ab-6376-4891-98ae-6942f62fc56b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.network.Port" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9fa9ab72-affd-45ba-939c-f5b19c965310 EntryTimestamp=2022-04-26T06:21:47.633Z InvocationID=04e65402-0b88-4f5c-afc8-6153c61a5be1 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 86f30a27-0ecb-457f-8831-b483e785ca45 2022-04-26T06:21:48.705Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "GlobalPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "9fa9ab72-affd-45ba-939c-f5b19c965310" INVARIANT_UUID = "087707ab-6376-4891-98ae-6942f62fc56b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "86f30a27-0ecb-457f-8831-b483e785ca45.assettoscatemplate" CURR_ARTIFACT_UUID = "86f30a27-0ecb-457f-8831-b483e785ca45.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Globalport-template.yml,0,86f30a27-0ecb-457f-8831-b483e785ca45.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=1072 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=84a8c77d-c206-48d9-bfc2-da9624624c43 EntryTimestamp=2022-04-26T06:21:47.633Z InvocationID=04e65402-0b88-4f5c-afc8-6153c61a5be1 LogTimestamp=2022-04-26T06:21:48.705188Z ResponseDesc=Created Exiting. 2022-04-26T06:21:48.706Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:21:48.733Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=14c0bd8b-6fc5-402d-933b-9c776b7d6bf0 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:48.733338Z InvocationID=8f9e995e-585d-4d4e-a078-9271d48c1a38 LogTimestamp=2022-04-26T06:21:48.733438Z Entering 2022-04-26T06:21:48.734Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=14c0bd8b-6fc5-402d-933b-9c776b7d6bf0 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:48.733338Z InvocationID=8f9e995e-585d-4d4e-a078-9271d48c1a38 LogTimestamp=2022-04-26T06:21:48.733438Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:21:48.854Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b29e3bf4-d30d-4a5e-9072-8d66ae10b08b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:21:48.854Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b29e3bf4-d30d-4a5e-9072-8d66ae10b08b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@30306fd1 2022-04-26T06:21:48.854Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b29e3bf4-d30d-4a5e-9072-8d66ae10b08b PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:21:48.854Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b29e3bf4-d30d-4a5e-9072-8d66ae10b08b PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:21:48.858Z [BE-Health-Check-Task] INFO o.a.http.impl.execchain.RetryExec RequestID=b29e3bf4-d30d-4a5e-9072-8d66ae10b08b PartnerName=SDC.BE I/O exception (org.apache.http.NoHttpResponseException) caught when processing request to {s}->https://sdc-onboarding-be.onap:8445: The target server failed to respond 2022-04-26T06:21:48.858Z [BE-Health-Check-Task] INFO o.a.http.impl.execchain.RetryExec RequestID=b29e3bf4-d30d-4a5e-9072-8d66ae10b08b PartnerName=SDC.BE Retrying request to {s}->https://sdc-onboarding-be.onap:8445 2022-04-26T06:21:48.995Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:48.734Z InvocationID=c60a39de-dd46-469d-a3a8-79f82fa3d4f9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:48.995Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:48.734Z InvocationID=c60a39de-dd46-469d-a3a8-79f82fa3d4f9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:48.995Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:48.734Z InvocationID=c60a39de-dd46-469d-a3a8-79f82fa3d4f9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:48.995Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:48.734Z InvocationID=c60a39de-dd46-469d-a3a8-79f82fa3d4f9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:49.003Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:48.734Z InvocationID=c60a39de-dd46-469d-a3a8-79f82fa3d4f9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Globalcompute 2022-04-26T06:21:49.118Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "GlobalCompute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "6219dbc2-6a42-4de9-9aa6-7d150f0e4528" INVARIANT_UUID = "828f96c3-2e17-4b2a-8743-55c0b1983d81" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.Compute" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6219dbc2-6a42-4de9-9aa6-7d150f0e4528 EntryTimestamp=2022-04-26T06:21:48.734Z InvocationID=c60a39de-dd46-469d-a3a8-79f82fa3d4f9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 0235cba2-0c0f-4085-b460-392d0d4826d0 2022-04-26T06:21:49.902Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=b29e3bf4-d30d-4a5e-9072-8d66ae10b08b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:21:49.903Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=b29e3bf4-d30d-4a5e-9072-8d66ae10b08b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:21:49.960Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "GlobalCompute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "6219dbc2-6a42-4de9-9aa6-7d150f0e4528" INVARIANT_UUID = "828f96c3-2e17-4b2a-8743-55c0b1983d81" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "0235cba2-0c0f-4085-b460-392d0d4826d0.assettoscatemplate" CURR_ARTIFACT_UUID = "0235cba2-0c0f-4085-b460-392d0d4826d0.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Globalcompute-template.yml,0,0235cba2-0c0f-4085-b460-392d0d4826d0.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=1226 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e67deb81-d150-400c-a477-87fa855cbed7 EntryTimestamp=2022-04-26T06:21:48.734Z InvocationID=c60a39de-dd46-469d-a3a8-79f82fa3d4f9 LogTimestamp=2022-04-26T06:21:49.960292Z ResponseDesc=Created Exiting. 2022-04-26T06:21:49.961Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:21:49.978Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a18b743a-94c7-4462-af92-1a0004a0c23e InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:49.978060Z InvocationID=c25ef59c-f0df-4bcf-bda9-4a5a398671e3 LogTimestamp=2022-04-26T06:21:49.978211Z Entering 2022-04-26T06:21:49.979Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a18b743a-94c7-4462-af92-1a0004a0c23e InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:49.978060Z InvocationID=c25ef59c-f0df-4bcf-bda9-4a5a398671e3 LogTimestamp=2022-04-26T06:21:49.978211Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:21:50.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:21:50.378Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:21:50.549Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:49.980Z InvocationID=9c2dd508-a6fc-4f7f-91eb-df00f257ef3a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:50.549Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:49.980Z InvocationID=9c2dd508-a6fc-4f7f-91eb-df00f257ef3a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:50.549Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:49.980Z InvocationID=9c2dd508-a6fc-4f7f-91eb-df00f257ef3a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:50.550Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:49.980Z InvocationID=9c2dd508-a6fc-4f7f-91eb-df00f257ef3a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:50.550Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:49.980Z InvocationID=9c2dd508-a6fc-4f7f-91eb-df00f257ef3a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Volume 2022-04-26T06:21:50.848Z [qtp2085002312-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "volume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e9dbe22c-8b40-47e8-86fe-f06079edf331" INVARIANT_UUID = "8448b2fb-afa1-495c-ac98-fa9895e22e07" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.volume" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e9dbe22c-8b40-47e8-86fe-f06079edf331 EntryTimestamp=2022-04-26T06:21:49.980Z InvocationID=9c2dd508-a6fc-4f7f-91eb-df00f257ef3a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:21:50.849Z [qtp2085002312-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "volume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e9dbe22c-8b40-47e8-86fe-f06079edf331" INVARIANT_UUID = "8448b2fb-afa1-495c-ac98-fa9895e22e07" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.volume" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e9dbe22c-8b40-47e8-86fe-f06079edf331 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:49.980Z InvocationID=9c2dd508-a6fc-4f7f-91eb-df00f257ef3a ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:21:51.061Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "volume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e9dbe22c-8b40-47e8-86fe-f06079edf331" INVARIANT_UUID = "8448b2fb-afa1-495c-ac98-fa9895e22e07" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.volume" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e9dbe22c-8b40-47e8-86fe-f06079edf331 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:49.980Z InvocationID=9c2dd508-a6fc-4f7f-91eb-df00f257ef3a ErrorCode=500 lock resource with id 33510c50-2515-4c77-92ef-2820052fb80f 2022-04-26T06:21:51.940Z [ComponentsCleanThread-1] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null lock resource with name DELETE_LOCKER 2022-04-26T06:21:51.941Z [ComponentsCleanThread-1] INFO o.o.s.b.c.s.ComponentsCleanBusinessLogic RequestID=null Lock cleanup operation is done with result = OK 2022-04-26T06:21:51.945Z [ComponentsCleanThread-1] INFO o.o.s.b.c.s.ComponentsCleanBusinessLogic RequestID=null Unlock cleanup operation is done with result = OK 2022-04-26T06:21:52.350Z [qtp2085002312-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "volume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e9dbe22c-8b40-47e8-86fe-f06079edf331" INVARIANT_UUID = "8448b2fb-afa1-495c-ac98-fa9895e22e07" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "33510c50-2515-4c77-92ef-2820052fb80f.assettoscatemplate" CURR_ARTIFACT_UUID = "33510c50-2515-4c77-92ef-2820052fb80f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Volume-template.yml,0,33510c50-2515-4c77-92ef-2820052fb80f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=461ac7ac-d584-49c1-ac9b-da1dd6386fec TargetServiceName=null EntryTimestamp=2022-04-26T06:21:49.980Z InvocationID=9c2dd508-a6fc-4f7f-91eb-df00f257ef3a ErrorCode=500 Starting to create Csar for component volume 2022-04-26T06:21:52.547Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12f911ef-c3b1-4544-9f44-c57240a5bd31 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:21:52.547Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12f911ef-c3b1-4544-9f44-c57240a5bd31 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@38cbc7b 2022-04-26T06:21:52.547Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12f911ef-c3b1-4544-9f44-c57240a5bd31 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:21:52.547Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12f911ef-c3b1-4544-9f44-c57240a5bd31 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:21:52.565Z [qtp2085002312-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "volume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e9dbe22c-8b40-47e8-86fe-f06079edf331" INVARIANT_UUID = "8448b2fb-afa1-495c-ac98-fa9895e22e07" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "33510c50-2515-4c77-92ef-2820052fb80f.assettoscatemplate" CURR_ARTIFACT_UUID = "33510c50-2515-4c77-92ef-2820052fb80f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Volume-template.yml,0,33510c50-2515-4c77-92ef-2820052fb80f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=461ac7ac-d584-49c1-ac9b-da1dd6386fec TargetServiceName=null EntryTimestamp=2022-04-26T06:21:49.980Z InvocationID=9c2dd508-a6fc-4f7f-91eb-df00f257ef3a ErrorCode=500 Ended create Csar for component volume 2022-04-26T06:21:52.566Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "volume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e9dbe22c-8b40-47e8-86fe-f06079edf331" INVARIANT_UUID = "8448b2fb-afa1-495c-ac98-fa9895e22e07" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "33510c50-2515-4c77-92ef-2820052fb80f.assettoscatemplate" CURR_ARTIFACT_UUID = "33510c50-2515-4c77-92ef-2820052fb80f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Volume-template.yml,0,33510c50-2515-4c77-92ef-2820052fb80f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=461ac7ac-d584-49c1-ac9b-da1dd6386fec TargetServiceName=null EntryTimestamp=2022-04-26T06:21:49.980Z InvocationID=9c2dd508-a6fc-4f7f-91eb-df00f257ef3a ErrorCode=500 lock resource with id 33510c50-2515-4c77-92ef-2820052fb80f 2022-04-26T06:21:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:21:52.686Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:21:52.686Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:21:53.357Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "volume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e9dbe22c-8b40-47e8-86fe-f06079edf331" INVARIANT_UUID = "8448b2fb-afa1-495c-ac98-fa9895e22e07" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "33510c50-2515-4c77-92ef-2820052fb80f.assettoscacsar" CURR_ARTIFACT_UUID = "33510c50-2515-4c77-92ef-2820052fb80f.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Volume-csar.csar,0,33510c50-2515-4c77-92ef-2820052fb80f.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=3377 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4dd542c2-3e3c-49b4-8c97-dc368c511967 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:49.980Z InvocationID=9c2dd508-a6fc-4f7f-91eb-df00f257ef3a ErrorCode=500 LogTimestamp=2022-04-26T06:21:53.357268Z ResponseDesc=Created Exiting. 2022-04-26T06:21:53.358Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:21:53.372Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=3d25a852-ae08-4225-8281-95dda3c66e42 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:53.371828Z InvocationID=565e5b17-c7b7-4d94-b27a-433331d553d0 LogTimestamp=2022-04-26T06:21:53.371914Z Entering 2022-04-26T06:21:53.372Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=3d25a852-ae08-4225-8281-95dda3c66e42 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:53.371828Z InvocationID=565e5b17-c7b7-4d94-b27a-433331d553d0 LogTimestamp=2022-04-26T06:21:53.371914Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:21:53.581Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=12f911ef-c3b1-4544-9f44-c57240a5bd31 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:21:53.581Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=12f911ef-c3b1-4544-9f44-c57240a5bd31 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:21:54.364Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:53.372Z InvocationID=bf30cb18-2e9b-43bb-a5c0-bd14dad3e34e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:54.365Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:53.372Z InvocationID=bf30cb18-2e9b-43bb-a5c0-bd14dad3e34e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:54.365Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:53.372Z InvocationID=bf30cb18-2e9b-43bb-a5c0-bd14dad3e34e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:54.365Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:53.372Z InvocationID=bf30cb18-2e9b-43bb-a5c0-bd14dad3e34e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:54.366Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:53.372Z InvocationID=bf30cb18-2e9b-43bb-a5c0-bd14dad3e34e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Cindervolume 2022-04-26T06:21:54.844Z [qtp2085002312-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "CinderVolume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "527ec799-8620-40e7-98b0-56329b58e649" INVARIANT_UUID = "047fdbdc-72b9-4519-bfc9-3a558be6be7c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.heat.cinder.Volume" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=527ec799-8620-40e7-98b0-56329b58e649 EntryTimestamp=2022-04-26T06:21:53.372Z InvocationID=bf30cb18-2e9b-43bb-a5c0-bd14dad3e34e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:21:54.845Z [qtp2085002312-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "CinderVolume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "527ec799-8620-40e7-98b0-56329b58e649" INVARIANT_UUID = "047fdbdc-72b9-4519-bfc9-3a558be6be7c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.heat.cinder.Volume" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=527ec799-8620-40e7-98b0-56329b58e649 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:53.372Z InvocationID=bf30cb18-2e9b-43bb-a5c0-bd14dad3e34e ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:21:55.363Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "CinderVolume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "527ec799-8620-40e7-98b0-56329b58e649" INVARIANT_UUID = "047fdbdc-72b9-4519-bfc9-3a558be6be7c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.heat.cinder.Volume" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=527ec799-8620-40e7-98b0-56329b58e649 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:53.372Z InvocationID=bf30cb18-2e9b-43bb-a5c0-bd14dad3e34e ErrorCode=500 lock resource with id 22da776b-174e-4c3d-b547-5468c4d703a8 2022-04-26T06:21:55.853Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68c05b22-f628-4820-add1-4a71bfdd066a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:21:55.853Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68c05b22-f628-4820-add1-4a71bfdd066a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5b32e31 2022-04-26T06:21:55.853Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68c05b22-f628-4820-add1-4a71bfdd066a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:21:55.854Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68c05b22-f628-4820-add1-4a71bfdd066a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:21:56.651Z [qtp2085002312-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "CinderVolume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "527ec799-8620-40e7-98b0-56329b58e649" INVARIANT_UUID = "047fdbdc-72b9-4519-bfc9-3a558be6be7c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "22da776b-174e-4c3d-b547-5468c4d703a8.assettoscatemplate" CURR_ARTIFACT_UUID = "22da776b-174e-4c3d-b547-5468c4d703a8.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Cindervolume-template.yml,0,22da776b-174e-4c3d-b547-5468c4d703a8.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a267407b-0ab9-49bb-81ba-0755ea80dcf0 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:53.372Z InvocationID=bf30cb18-2e9b-43bb-a5c0-bd14dad3e34e ErrorCode=500 Starting to create Csar for component CinderVolume 2022-04-26T06:21:56.832Z [qtp2085002312-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "CinderVolume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "527ec799-8620-40e7-98b0-56329b58e649" INVARIANT_UUID = "047fdbdc-72b9-4519-bfc9-3a558be6be7c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "22da776b-174e-4c3d-b547-5468c4d703a8.assettoscatemplate" CURR_ARTIFACT_UUID = "22da776b-174e-4c3d-b547-5468c4d703a8.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Cindervolume-template.yml,0,22da776b-174e-4c3d-b547-5468c4d703a8.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a267407b-0ab9-49bb-81ba-0755ea80dcf0 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:53.372Z InvocationID=bf30cb18-2e9b-43bb-a5c0-bd14dad3e34e ErrorCode=500 Ended create Csar for component CinderVolume 2022-04-26T06:21:56.833Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "CinderVolume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "527ec799-8620-40e7-98b0-56329b58e649" INVARIANT_UUID = "047fdbdc-72b9-4519-bfc9-3a558be6be7c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "22da776b-174e-4c3d-b547-5468c4d703a8.assettoscatemplate" CURR_ARTIFACT_UUID = "22da776b-174e-4c3d-b547-5468c4d703a8.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Cindervolume-template.yml,0,22da776b-174e-4c3d-b547-5468c4d703a8.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a267407b-0ab9-49bb-81ba-0755ea80dcf0 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:53.372Z InvocationID=bf30cb18-2e9b-43bb-a5c0-bd14dad3e34e ErrorCode=500 lock resource with id 22da776b-174e-4c3d-b547-5468c4d703a8 2022-04-26T06:21:56.879Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=68c05b22-f628-4820-add1-4a71bfdd066a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:21:56.880Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=68c05b22-f628-4820-add1-4a71bfdd066a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:21:57.571Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "CinderVolume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "527ec799-8620-40e7-98b0-56329b58e649" INVARIANT_UUID = "047fdbdc-72b9-4519-bfc9-3a558be6be7c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "22da776b-174e-4c3d-b547-5468c4d703a8.assettoscacsar" CURR_ARTIFACT_UUID = "22da776b-174e-4c3d-b547-5468c4d703a8.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Cindervolume-csar.csar,0,22da776b-174e-4c3d-b547-5468c4d703a8.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=4198 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=234370d4-dccf-4379-988e-4b59e026324f TargetServiceName=null EntryTimestamp=2022-04-26T06:21:53.372Z InvocationID=bf30cb18-2e9b-43bb-a5c0-bd14dad3e34e ErrorCode=500 LogTimestamp=2022-04-26T06:21:57.570847Z ResponseDesc=Created Exiting. 2022-04-26T06:21:57.571Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:21:57.651Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a73de58b-7baa-412f-acc5-5c1693c3d6d5 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:57.650545Z InvocationID=e23b6b53-1920-4149-aacf-26f2b4c324e3 LogTimestamp=2022-04-26T06:21:57.650686Z Entering 2022-04-26T06:21:57.652Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a73de58b-7baa-412f-acc5-5c1693c3d6d5 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:21:57.650545Z InvocationID=e23b6b53-1920-4149-aacf-26f2b4c324e3 LogTimestamp=2022-04-26T06:21:57.650686Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:21:58.253Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c20c2db9-a426-4ad0-afc5-6d648dbbafe4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:21:58.253Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c20c2db9-a426-4ad0-afc5-6d648dbbafe4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2c908f81 2022-04-26T06:21:58.253Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c20c2db9-a426-4ad0-afc5-6d648dbbafe4 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:21:58.253Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c20c2db9-a426-4ad0-afc5-6d648dbbafe4 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:21:58.358Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:57.652Z InvocationID=33157467-8751-4d14-ad2f-9c57421a7980 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:58.358Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:57.652Z InvocationID=33157467-8751-4d14-ad2f-9c57421a7980 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:58.358Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:57.652Z InvocationID=33157467-8751-4d14-ad2f-9c57421a7980 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:58.358Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:57.652Z InvocationID=33157467-8751-4d14-ad2f-9c57421a7980 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:21:58.443Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:21:57.652Z InvocationID=33157467-8751-4d14-ad2f-9c57421a7980 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Contrailvirtualnetwork 2022-04-26T06:21:58.756Z [qtp2085002312-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailVirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "5329b6f3-7731-416e-a09d-559418b7e760" INVARIANT_UUID = "4e8e1f67-8b6d-4d42-bbbd-979dfee36065" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.nodes.heat.network.contrail.VirtualNetwork" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5329b6f3-7731-416e-a09d-559418b7e760 EntryTimestamp=2022-04-26T06:21:57.652Z InvocationID=33157467-8751-4d14-ad2f-9c57421a7980 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:21:58.756Z [qtp2085002312-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailVirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "5329b6f3-7731-416e-a09d-559418b7e760" INVARIANT_UUID = "4e8e1f67-8b6d-4d42-bbbd-979dfee36065" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.nodes.heat.network.contrail.VirtualNetwork" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5329b6f3-7731-416e-a09d-559418b7e760 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:57.652Z InvocationID=33157467-8751-4d14-ad2f-9c57421a7980 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:21:59.151Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailVirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "5329b6f3-7731-416e-a09d-559418b7e760" INVARIANT_UUID = "4e8e1f67-8b6d-4d42-bbbd-979dfee36065" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.nodes.heat.network.contrail.VirtualNetwork" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5329b6f3-7731-416e-a09d-559418b7e760 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:57.652Z InvocationID=33157467-8751-4d14-ad2f-9c57421a7980 ErrorCode=500 lock resource with id c7509b1f-c9fc-4401-b7ab-741720b662a4 2022-04-26T06:21:59.277Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c20c2db9-a426-4ad0-afc5-6d648dbbafe4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:21:59.277Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c20c2db9-a426-4ad0-afc5-6d648dbbafe4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:22:01.827Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85999820-47f2-42ec-a861-0026cb47f20f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:22:01.828Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85999820-47f2-42ec-a861-0026cb47f20f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@38ec5785 2022-04-26T06:22:01.828Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85999820-47f2-42ec-a861-0026cb47f20f PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:22:01.828Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85999820-47f2-42ec-a861-0026cb47f20f PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:22:02.143Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailVirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "5329b6f3-7731-416e-a09d-559418b7e760" INVARIANT_UUID = "4e8e1f67-8b6d-4d42-bbbd-979dfee36065" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c7509b1f-c9fc-4401-b7ab-741720b662a4.assettoscatemplate" CURR_ARTIFACT_UUID = "c7509b1f-c9fc-4401-b7ab-741720b662a4.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailvirtualnetwork-template.yml,0,c7509b1f-c9fc-4401-b7ab-741720b662a4.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b330d686-7f59-4887-95ed-e07c67a11534 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:57.652Z InvocationID=33157467-8751-4d14-ad2f-9c57421a7980 ErrorCode=500 Starting to create Csar for component ContrailVirtualNetwork 2022-04-26T06:22:02.390Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailVirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "5329b6f3-7731-416e-a09d-559418b7e760" INVARIANT_UUID = "4e8e1f67-8b6d-4d42-bbbd-979dfee36065" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c7509b1f-c9fc-4401-b7ab-741720b662a4.assettoscatemplate" CURR_ARTIFACT_UUID = "c7509b1f-c9fc-4401-b7ab-741720b662a4.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailvirtualnetwork-template.yml,0,c7509b1f-c9fc-4401-b7ab-741720b662a4.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b330d686-7f59-4887-95ed-e07c67a11534 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:57.652Z InvocationID=33157467-8751-4d14-ad2f-9c57421a7980 ErrorCode=500 Ended create Csar for component ContrailVirtualNetwork 2022-04-26T06:22:02.391Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailVirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "5329b6f3-7731-416e-a09d-559418b7e760" INVARIANT_UUID = "4e8e1f67-8b6d-4d42-bbbd-979dfee36065" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c7509b1f-c9fc-4401-b7ab-741720b662a4.assettoscatemplate" CURR_ARTIFACT_UUID = "c7509b1f-c9fc-4401-b7ab-741720b662a4.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailvirtualnetwork-template.yml,0,c7509b1f-c9fc-4401-b7ab-741720b662a4.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b330d686-7f59-4887-95ed-e07c67a11534 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:57.652Z InvocationID=33157467-8751-4d14-ad2f-9c57421a7980 ErrorCode=500 lock resource with id c7509b1f-c9fc-4401-b7ab-741720b662a4 2022-04-26T06:22:02.893Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=85999820-47f2-42ec-a861-0026cb47f20f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:22:02.894Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=85999820-47f2-42ec-a861-0026cb47f20f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:22:03.452Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailVirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "5329b6f3-7731-416e-a09d-559418b7e760" INVARIANT_UUID = "4e8e1f67-8b6d-4d42-bbbd-979dfee36065" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c7509b1f-c9fc-4401-b7ab-741720b662a4.assettoscacsar" CURR_ARTIFACT_UUID = "c7509b1f-c9fc-4401-b7ab-741720b662a4.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Contrailvirtualnetwork-csar.csar,0,c7509b1f-c9fc-4401-b7ab-741720b662a4.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5800 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ed0a977f-0aa3-4c20-accf-14d3d4282d91 TargetServiceName=null EntryTimestamp=2022-04-26T06:21:57.652Z InvocationID=33157467-8751-4d14-ad2f-9c57421a7980 ErrorCode=500 LogTimestamp=2022-04-26T06:22:03.452225Z ResponseDesc=Created Exiting. 2022-04-26T06:22:03.453Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:22:03.543Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=b506c656-a105-4d44-b4f3-dee3928e3da9 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:03.543471Z InvocationID=b5c86e0a-7f6b-4ae3-b16d-dd7e57d1ab43 LogTimestamp=2022-04-26T06:22:03.543571Z Entering 2022-04-26T06:22:03.544Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=b506c656-a105-4d44-b4f3-dee3928e3da9 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:03.543471Z InvocationID=b5c86e0a-7f6b-4ae3-b16d-dd7e57d1ab43 LogTimestamp=2022-04-26T06:22:03.543571Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:22:04.437Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba1b2259-7340-4308-a60f-460117abd526 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:22:04.437Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba1b2259-7340-4308-a60f-460117abd526 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1659bef5 2022-04-26T06:22:04.437Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba1b2259-7340-4308-a60f-460117abd526 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:22:04.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba1b2259-7340-4308-a60f-460117abd526 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:22:05.058Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:03.544Z InvocationID=2c65192f-7ddd-44b4-a2a2-5fc6496f3bed PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:05.058Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:03.544Z InvocationID=2c65192f-7ddd-44b4-a2a2-5fc6496f3bed PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:05.058Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:03.544Z InvocationID=2c65192f-7ddd-44b4-a2a2-5fc6496f3bed PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:05.058Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:03.544Z InvocationID=2c65192f-7ddd-44b4-a2a2-5fc6496f3bed PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:05.143Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:03.544Z InvocationID=2c65192f-7ddd-44b4-a2a2-5fc6496f3bed PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Neutronnet 2022-04-26T06:22:05.352Z [qtp2085002312-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "39bc2b02-4306-4551-b178-c17e3a6f4f03" INVARIANT_UUID = "67b21702-ad6c-432c-82f2-6bbb64ced013" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.nodes.heat.network.neutron.Net" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=39bc2b02-4306-4551-b178-c17e3a6f4f03 EntryTimestamp=2022-04-26T06:22:03.544Z InvocationID=2c65192f-7ddd-44b4-a2a2-5fc6496f3bed PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:22:05.352Z [qtp2085002312-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "39bc2b02-4306-4551-b178-c17e3a6f4f03" INVARIANT_UUID = "67b21702-ad6c-432c-82f2-6bbb64ced013" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.nodes.heat.network.neutron.Net" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=39bc2b02-4306-4551-b178-c17e3a6f4f03 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:03.544Z InvocationID=2c65192f-7ddd-44b4-a2a2-5fc6496f3bed ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:22:05.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:22:05.381Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:22:05.449Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ba1b2259-7340-4308-a60f-460117abd526 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:22:05.450Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ba1b2259-7340-4308-a60f-460117abd526 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:22:05.558Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "39bc2b02-4306-4551-b178-c17e3a6f4f03" INVARIANT_UUID = "67b21702-ad6c-432c-82f2-6bbb64ced013" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.nodes.heat.network.neutron.Net" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=39bc2b02-4306-4551-b178-c17e3a6f4f03 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:03.544Z InvocationID=2c65192f-7ddd-44b4-a2a2-5fc6496f3bed ErrorCode=500 lock resource with id 6e7bab2b-edf2-4ca1-936e-21ec49149e80 2022-04-26T06:22:07.529Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9bd8ebd6-364c-4aa0-9947-4d5a0503b1b9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:22:07.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9bd8ebd6-364c-4aa0-9947-4d5a0503b1b9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@79689e2b 2022-04-26T06:22:07.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9bd8ebd6-364c-4aa0-9947-4d5a0503b1b9 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:22:07.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9bd8ebd6-364c-4aa0-9947-4d5a0503b1b9 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:22:07.644Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:22:08.235Z [qtp2085002312-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "39bc2b02-4306-4551-b178-c17e3a6f4f03" INVARIANT_UUID = "67b21702-ad6c-432c-82f2-6bbb64ced013" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "6e7bab2b-edf2-4ca1-936e-21ec49149e80.assettoscatemplate" CURR_ARTIFACT_UUID = "6e7bab2b-edf2-4ca1-936e-21ec49149e80.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Neutronnet-template.yml,0,6e7bab2b-edf2-4ca1-936e-21ec49149e80.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3045632d-5cac-4835-ad92-ce554bf19fb0 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:03.544Z InvocationID=2c65192f-7ddd-44b4-a2a2-5fc6496f3bed ErrorCode=500 Starting to create Csar for component NeutronNet 2022-04-26T06:22:08.419Z [qtp2085002312-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "39bc2b02-4306-4551-b178-c17e3a6f4f03" INVARIANT_UUID = "67b21702-ad6c-432c-82f2-6bbb64ced013" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "6e7bab2b-edf2-4ca1-936e-21ec49149e80.assettoscatemplate" CURR_ARTIFACT_UUID = "6e7bab2b-edf2-4ca1-936e-21ec49149e80.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Neutronnet-template.yml,0,6e7bab2b-edf2-4ca1-936e-21ec49149e80.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3045632d-5cac-4835-ad92-ce554bf19fb0 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:03.544Z InvocationID=2c65192f-7ddd-44b4-a2a2-5fc6496f3bed ErrorCode=500 Ended create Csar for component NeutronNet 2022-04-26T06:22:08.419Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "39bc2b02-4306-4551-b178-c17e3a6f4f03" INVARIANT_UUID = "67b21702-ad6c-432c-82f2-6bbb64ced013" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "6e7bab2b-edf2-4ca1-936e-21ec49149e80.assettoscatemplate" CURR_ARTIFACT_UUID = "6e7bab2b-edf2-4ca1-936e-21ec49149e80.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Neutronnet-template.yml,0,6e7bab2b-edf2-4ca1-936e-21ec49149e80.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3045632d-5cac-4835-ad92-ce554bf19fb0 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:03.544Z InvocationID=2c65192f-7ddd-44b4-a2a2-5fc6496f3bed ErrorCode=500 lock resource with id 6e7bab2b-edf2-4ca1-936e-21ec49149e80 2022-04-26T06:22:08.557Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=9bd8ebd6-364c-4aa0-9947-4d5a0503b1b9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:22:08.558Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=9bd8ebd6-364c-4aa0-9947-4d5a0503b1b9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:22:09.465Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "39bc2b02-4306-4551-b178-c17e3a6f4f03" INVARIANT_UUID = "67b21702-ad6c-432c-82f2-6bbb64ced013" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "6e7bab2b-edf2-4ca1-936e-21ec49149e80.assettoscacsar" CURR_ARTIFACT_UUID = "6e7bab2b-edf2-4ca1-936e-21ec49149e80.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Neutronnet-csar.csar,0,6e7bab2b-edf2-4ca1-936e-21ec49149e80.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5920 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=99981a4a-53c5-489f-b85b-b9fb57d3daad TargetServiceName=null EntryTimestamp=2022-04-26T06:22:03.544Z InvocationID=2c65192f-7ddd-44b4-a2a2-5fc6496f3bed ErrorCode=500 LogTimestamp=2022-04-26T06:22:09.464698Z ResponseDesc=Created Exiting. 2022-04-26T06:22:09.466Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:22:09.536Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=09971537-2fd1-4418-9408-a99b1e59d6dc InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:09.534093Z InvocationID=aa10b232-5767-47e4-ab3a-fd32820b361f LogTimestamp=2022-04-26T06:22:09.534808Z Entering 2022-04-26T06:22:09.538Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=09971537-2fd1-4418-9408-a99b1e59d6dc InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:09.534093Z InvocationID=aa10b232-5767-47e4-ab3a-fd32820b361f LogTimestamp=2022-04-26T06:22:09.534808Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:22:10.145Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50dd98b1-a0cc-45c8-ba6f-fcdcd5bb7d4e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:22:10.145Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50dd98b1-a0cc-45c8-ba6f-fcdcd5bb7d4e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@724162b1 2022-04-26T06:22:10.145Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50dd98b1-a0cc-45c8-ba6f-fcdcd5bb7d4e PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:22:10.145Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50dd98b1-a0cc-45c8-ba6f-fcdcd5bb7d4e PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:22:10.181Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:09.539Z InvocationID=b933eba3-6e01-408e-80f1-7c894dca38fc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:10.182Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:09.539Z InvocationID=b933eba3-6e01-408e-80f1-7c894dca38fc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:10.182Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:09.539Z InvocationID=b933eba3-6e01-408e-80f1-7c894dca38fc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:10.182Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:09.539Z InvocationID=b933eba3-6e01-408e-80f1-7c894dca38fc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:10.247Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:09.539Z InvocationID=b933eba3-6e01-408e-80f1-7c894dca38fc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Neutronport 2022-04-26T06:22:10.458Z [qtp2085002312-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NeutronPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "754b03e4-e4c0-4b7c-919a-87eb3a453420" INVARIANT_UUID = "57fa57b1-41bb-4dd7-b86a-52ce9aa38269" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.network.neutron.Port" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=754b03e4-e4c0-4b7c-919a-87eb3a453420 EntryTimestamp=2022-04-26T06:22:09.539Z InvocationID=b933eba3-6e01-408e-80f1-7c894dca38fc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:22:10.459Z [qtp2085002312-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NeutronPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "754b03e4-e4c0-4b7c-919a-87eb3a453420" INVARIANT_UUID = "57fa57b1-41bb-4dd7-b86a-52ce9aa38269" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.network.neutron.Port" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=754b03e4-e4c0-4b7c-919a-87eb3a453420 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:09.539Z InvocationID=b933eba3-6e01-408e-80f1-7c894dca38fc ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:22:10.558Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NeutronPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "754b03e4-e4c0-4b7c-919a-87eb3a453420" INVARIANT_UUID = "57fa57b1-41bb-4dd7-b86a-52ce9aa38269" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.network.neutron.Port" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=754b03e4-e4c0-4b7c-919a-87eb3a453420 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:09.539Z InvocationID=b933eba3-6e01-408e-80f1-7c894dca38fc ErrorCode=500 lock resource with id 1be316be-04cb-409f-bb85-8d77c01dd269 2022-04-26T06:22:11.185Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=50dd98b1-a0cc-45c8-ba6f-fcdcd5bb7d4e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:22:11.185Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=50dd98b1-a0cc-45c8-ba6f-fcdcd5bb7d4e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:22:11.447Z [qtp2085002312-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NeutronPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "754b03e4-e4c0-4b7c-919a-87eb3a453420" INVARIANT_UUID = "57fa57b1-41bb-4dd7-b86a-52ce9aa38269" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1be316be-04cb-409f-bb85-8d77c01dd269.assettoscatemplate" CURR_ARTIFACT_UUID = "1be316be-04cb-409f-bb85-8d77c01dd269.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Neutronport-template.yml,0,1be316be-04cb-409f-bb85-8d77c01dd269.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0d76e086-2065-4f9e-8b81-f199a2bc24e1 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:09.539Z InvocationID=b933eba3-6e01-408e-80f1-7c894dca38fc ErrorCode=500 Starting to create Csar for component NeutronPort 2022-04-26T06:22:11.609Z [qtp2085002312-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NeutronPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "754b03e4-e4c0-4b7c-919a-87eb3a453420" INVARIANT_UUID = "57fa57b1-41bb-4dd7-b86a-52ce9aa38269" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1be316be-04cb-409f-bb85-8d77c01dd269.assettoscatemplate" CURR_ARTIFACT_UUID = "1be316be-04cb-409f-bb85-8d77c01dd269.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Neutronport-template.yml,0,1be316be-04cb-409f-bb85-8d77c01dd269.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0d76e086-2065-4f9e-8b81-f199a2bc24e1 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:09.539Z InvocationID=b933eba3-6e01-408e-80f1-7c894dca38fc ErrorCode=500 Ended create Csar for component NeutronPort 2022-04-26T06:22:11.610Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NeutronPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "754b03e4-e4c0-4b7c-919a-87eb3a453420" INVARIANT_UUID = "57fa57b1-41bb-4dd7-b86a-52ce9aa38269" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1be316be-04cb-409f-bb85-8d77c01dd269.assettoscatemplate" CURR_ARTIFACT_UUID = "1be316be-04cb-409f-bb85-8d77c01dd269.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Neutronport-template.yml,0,1be316be-04cb-409f-bb85-8d77c01dd269.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0d76e086-2065-4f9e-8b81-f199a2bc24e1 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:09.539Z InvocationID=b933eba3-6e01-408e-80f1-7c894dca38fc ErrorCode=500 lock resource with id 1be316be-04cb-409f-bb85-8d77c01dd269 2022-04-26T06:22:12.187Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NeutronPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "754b03e4-e4c0-4b7c-919a-87eb3a453420" INVARIANT_UUID = "57fa57b1-41bb-4dd7-b86a-52ce9aa38269" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1be316be-04cb-409f-bb85-8d77c01dd269.assettoscacsar" CURR_ARTIFACT_UUID = "1be316be-04cb-409f-bb85-8d77c01dd269.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Neutronport-csar.csar,0,1be316be-04cb-409f-bb85-8d77c01dd269.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=2647 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e0d7f7c7-d0e0-4906-9cae-b4e864a01060 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:09.539Z InvocationID=b933eba3-6e01-408e-80f1-7c894dca38fc ErrorCode=500 LogTimestamp=2022-04-26T06:22:12.186922Z ResponseDesc=Created Exiting. 2022-04-26T06:22:12.187Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:22:12.232Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=8bd9d7be-fcbb-46b4-83e4-8035258d3da9 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:12.232378Z InvocationID=c72a0b03-97eb-4903-9ae7-0ed55abbd7df LogTimestamp=2022-04-26T06:22:12.232506Z Entering 2022-04-26T06:22:12.233Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=8bd9d7be-fcbb-46b4-83e4-8035258d3da9 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:12.232378Z InvocationID=c72a0b03-97eb-4903-9ae7-0ed55abbd7df LogTimestamp=2022-04-26T06:22:12.232506Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:22:12.383Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:12.233Z InvocationID=22164d9e-e538-4b8b-9a18-c18227c8c740 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:12.384Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:12.233Z InvocationID=22164d9e-e538-4b8b-9a18-c18227c8c740 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:12.384Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:12.233Z InvocationID=22164d9e-e538-4b8b-9a18-c18227c8c740 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:12.385Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:12.233Z InvocationID=22164d9e-e538-4b8b-9a18-c18227c8c740 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:12.385Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:12.233Z InvocationID=22164d9e-e538-4b8b-9a18-c18227c8c740 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Novaserver 2022-04-26T06:22:12.434Z [qtp2085002312-20] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NovaServer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2960db73-0bbd-4749-b532-1a14e64155de" INVARIANT_UUID = "71e80051-8cd3-4c37-93ad-fe5719427624" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.heat.nova.Server" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2960db73-0bbd-4749-b532-1a14e64155de EntryTimestamp=2022-04-26T06:22:12.233Z InvocationID=22164d9e-e538-4b8b-9a18-c18227c8c740 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:22:12.435Z [qtp2085002312-20] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NovaServer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2960db73-0bbd-4749-b532-1a14e64155de" INVARIANT_UUID = "71e80051-8cd3-4c37-93ad-fe5719427624" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.heat.nova.Server" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2960db73-0bbd-4749-b532-1a14e64155de TargetServiceName=null EntryTimestamp=2022-04-26T06:22:12.233Z InvocationID=22164d9e-e538-4b8b-9a18-c18227c8c740 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:22:12.488Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NovaServer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2960db73-0bbd-4749-b532-1a14e64155de" INVARIANT_UUID = "71e80051-8cd3-4c37-93ad-fe5719427624" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.heat.nova.Server" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2960db73-0bbd-4749-b532-1a14e64155de TargetServiceName=null EntryTimestamp=2022-04-26T06:22:12.233Z InvocationID=22164d9e-e538-4b8b-9a18-c18227c8c740 ErrorCode=500 lock resource with id d9533288-efb7-4c6a-8fba-3f9bf731779a 2022-04-26T06:22:12.792Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c613ada4-efea-454b-b350-442218340d64 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:22:12.792Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c613ada4-efea-454b-b350-442218340d64 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@799c29f3 2022-04-26T06:22:12.792Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c613ada4-efea-454b-b350-442218340d64 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:22:12.792Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c613ada4-efea-454b-b350-442218340d64 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:22:13.163Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NovaServer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2960db73-0bbd-4749-b532-1a14e64155de" INVARIANT_UUID = "71e80051-8cd3-4c37-93ad-fe5719427624" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "d9533288-efb7-4c6a-8fba-3f9bf731779a.assettoscatemplate" CURR_ARTIFACT_UUID = "d9533288-efb7-4c6a-8fba-3f9bf731779a.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Novaserver-template.yml,0,d9533288-efb7-4c6a-8fba-3f9bf731779a.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=dbb81b62-fba5-42a7-ba9f-8ce0f62be7db TargetServiceName=null EntryTimestamp=2022-04-26T06:22:12.233Z InvocationID=22164d9e-e538-4b8b-9a18-c18227c8c740 ErrorCode=500 Starting to create Csar for component NovaServer 2022-04-26T06:22:13.311Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NovaServer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2960db73-0bbd-4749-b532-1a14e64155de" INVARIANT_UUID = "71e80051-8cd3-4c37-93ad-fe5719427624" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "d9533288-efb7-4c6a-8fba-3f9bf731779a.assettoscatemplate" CURR_ARTIFACT_UUID = "d9533288-efb7-4c6a-8fba-3f9bf731779a.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Novaserver-template.yml,0,d9533288-efb7-4c6a-8fba-3f9bf731779a.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=dbb81b62-fba5-42a7-ba9f-8ce0f62be7db TargetServiceName=null EntryTimestamp=2022-04-26T06:22:12.233Z InvocationID=22164d9e-e538-4b8b-9a18-c18227c8c740 ErrorCode=500 Ended create Csar for component NovaServer 2022-04-26T06:22:13.312Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NovaServer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2960db73-0bbd-4749-b532-1a14e64155de" INVARIANT_UUID = "71e80051-8cd3-4c37-93ad-fe5719427624" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "d9533288-efb7-4c6a-8fba-3f9bf731779a.assettoscatemplate" CURR_ARTIFACT_UUID = "d9533288-efb7-4c6a-8fba-3f9bf731779a.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Novaserver-template.yml,0,d9533288-efb7-4c6a-8fba-3f9bf731779a.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=dbb81b62-fba5-42a7-ba9f-8ce0f62be7db TargetServiceName=null EntryTimestamp=2022-04-26T06:22:12.233Z InvocationID=22164d9e-e538-4b8b-9a18-c18227c8c740 ErrorCode=500 lock resource with id d9533288-efb7-4c6a-8fba-3f9bf731779a 2022-04-26T06:22:13.805Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c613ada4-efea-454b-b350-442218340d64 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:22:13.805Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c613ada4-efea-454b-b350-442218340d64 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:22:13.893Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NovaServer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2960db73-0bbd-4749-b532-1a14e64155de" INVARIANT_UUID = "71e80051-8cd3-4c37-93ad-fe5719427624" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "d9533288-efb7-4c6a-8fba-3f9bf731779a.assettoscacsar" CURR_ARTIFACT_UUID = "d9533288-efb7-4c6a-8fba-3f9bf731779a.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Novaserver-csar.csar,0,d9533288-efb7-4c6a-8fba-3f9bf731779a.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1659 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ba02627b-6128-43d2-b94a-01ce0651e002 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:12.233Z InvocationID=22164d9e-e538-4b8b-9a18-c18227c8c740 ErrorCode=500 LogTimestamp=2022-04-26T06:22:13.892821Z ResponseDesc=Created Exiting. 2022-04-26T06:22:13.893Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:22:13.913Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c195f3e2-8dbb-4497-9bde-15736cc05f3b InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:13.912923Z InvocationID=0e147896-c2e2-42cf-b6be-2992ff88c7b3 LogTimestamp=2022-04-26T06:22:13.913053Z Entering 2022-04-26T06:22:13.913Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c195f3e2-8dbb-4497-9bde-15736cc05f3b InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:13.912923Z InvocationID=0e147896-c2e2-42cf-b6be-2992ff88c7b3 LogTimestamp=2022-04-26T06:22:13.913053Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:22:14.144Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:13.914Z InvocationID=2810e27c-9a20-481e-93b8-55ca3afb970a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:14.144Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:13.914Z InvocationID=2810e27c-9a20-481e-93b8-55ca3afb970a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:14.144Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:13.914Z InvocationID=2810e27c-9a20-481e-93b8-55ca3afb970a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:14.144Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:13.914Z InvocationID=2810e27c-9a20-481e-93b8-55ca3afb970a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:14.148Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:13.914Z InvocationID=2810e27c-9a20-481e-93b8-55ca3afb970a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Extvl 2022-04-26T06:22:14.210Z [qtp2085002312-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ExtVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "f80f6268-179c-4ba6-99e9-07da3e6ca0e7" INVARIANT_UUID = "33ee3190-32f5-4cfc-a422-18e9361b39d5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.extVL" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f80f6268-179c-4ba6-99e9-07da3e6ca0e7 EntryTimestamp=2022-04-26T06:22:13.914Z InvocationID=2810e27c-9a20-481e-93b8-55ca3afb970a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:22:14.211Z [qtp2085002312-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ExtVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "f80f6268-179c-4ba6-99e9-07da3e6ca0e7" INVARIANT_UUID = "33ee3190-32f5-4cfc-a422-18e9361b39d5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.extVL" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f80f6268-179c-4ba6-99e9-07da3e6ca0e7 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:13.914Z InvocationID=2810e27c-9a20-481e-93b8-55ca3afb970a ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:22:14.256Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ExtVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "f80f6268-179c-4ba6-99e9-07da3e6ca0e7" INVARIANT_UUID = "33ee3190-32f5-4cfc-a422-18e9361b39d5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.extVL" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f80f6268-179c-4ba6-99e9-07da3e6ca0e7 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:13.914Z InvocationID=2810e27c-9a20-481e-93b8-55ca3afb970a ErrorCode=500 lock resource with id 59e73587-1bab-462c-9aad-4abbf7036ca4 2022-04-26T06:22:14.893Z [qtp2085002312-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ExtVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "f80f6268-179c-4ba6-99e9-07da3e6ca0e7" INVARIANT_UUID = "33ee3190-32f5-4cfc-a422-18e9361b39d5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "59e73587-1bab-462c-9aad-4abbf7036ca4.assettoscatemplate" CURR_ARTIFACT_UUID = "59e73587-1bab-462c-9aad-4abbf7036ca4.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extvl-template.yml,0,59e73587-1bab-462c-9aad-4abbf7036ca4.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=eed20266-f6dd-4382-baca-37b1c965087d TargetServiceName=null EntryTimestamp=2022-04-26T06:22:13.914Z InvocationID=2810e27c-9a20-481e-93b8-55ca3afb970a ErrorCode=500 Starting to create Csar for component ExtVL 2022-04-26T06:22:15.206Z [qtp2085002312-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ExtVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "f80f6268-179c-4ba6-99e9-07da3e6ca0e7" INVARIANT_UUID = "33ee3190-32f5-4cfc-a422-18e9361b39d5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "59e73587-1bab-462c-9aad-4abbf7036ca4.assettoscatemplate" CURR_ARTIFACT_UUID = "59e73587-1bab-462c-9aad-4abbf7036ca4.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extvl-template.yml,0,59e73587-1bab-462c-9aad-4abbf7036ca4.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=eed20266-f6dd-4382-baca-37b1c965087d TargetServiceName=null EntryTimestamp=2022-04-26T06:22:13.914Z InvocationID=2810e27c-9a20-481e-93b8-55ca3afb970a ErrorCode=500 Ended create Csar for component ExtVL 2022-04-26T06:22:15.207Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ExtVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "f80f6268-179c-4ba6-99e9-07da3e6ca0e7" INVARIANT_UUID = "33ee3190-32f5-4cfc-a422-18e9361b39d5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "59e73587-1bab-462c-9aad-4abbf7036ca4.assettoscatemplate" CURR_ARTIFACT_UUID = "59e73587-1bab-462c-9aad-4abbf7036ca4.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extvl-template.yml,0,59e73587-1bab-462c-9aad-4abbf7036ca4.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=eed20266-f6dd-4382-baca-37b1c965087d TargetServiceName=null EntryTimestamp=2022-04-26T06:22:13.914Z InvocationID=2810e27c-9a20-481e-93b8-55ca3afb970a ErrorCode=500 lock resource with id 59e73587-1bab-462c-9aad-4abbf7036ca4 2022-04-26T06:22:15.788Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ExtVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "f80f6268-179c-4ba6-99e9-07da3e6ca0e7" INVARIANT_UUID = "33ee3190-32f5-4cfc-a422-18e9361b39d5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "59e73587-1bab-462c-9aad-4abbf7036ca4.assettoscacsar" CURR_ARTIFACT_UUID = "59e73587-1bab-462c-9aad-4abbf7036ca4.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Extvl-csar.csar,0,59e73587-1bab-462c-9aad-4abbf7036ca4.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1873 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3494d01e-f64e-46fe-a75e-83d8569be455 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:13.914Z InvocationID=2810e27c-9a20-481e-93b8-55ca3afb970a ErrorCode=500 LogTimestamp=2022-04-26T06:22:15.787692Z ResponseDesc=Created Exiting. 2022-04-26T06:22:15.788Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:22:15.792Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c64ab8f2-34fa-484c-8a89-14eec4254728 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:22:15.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c64ab8f2-34fa-484c-8a89-14eec4254728 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@f71364b 2022-04-26T06:22:15.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c64ab8f2-34fa-484c-8a89-14eec4254728 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:22:15.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c64ab8f2-34fa-484c-8a89-14eec4254728 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:22:15.808Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=4e3d54d2-ffdd-4cbf-90e3-51e012220dbf InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:15.808270Z InvocationID=d7b812a0-547a-461a-995d-5c8c2360a872 LogTimestamp=2022-04-26T06:22:15.808363Z Entering 2022-04-26T06:22:15.809Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=4e3d54d2-ffdd-4cbf-90e3-51e012220dbf InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:15.808270Z InvocationID=d7b812a0-547a-461a-995d-5c8c2360a872 LogTimestamp=2022-04-26T06:22:15.808363Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:22:15.946Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:15.809Z InvocationID=5b982fe3-2322-48cd-a402-b4321af9c1d8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:15.946Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:15.809Z InvocationID=5b982fe3-2322-48cd-a402-b4321af9c1d8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:15.948Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:15.809Z InvocationID=5b982fe3-2322-48cd-a402-b4321af9c1d8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:15.948Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:15.809Z InvocationID=5b982fe3-2322-48cd-a402-b4321af9c1d8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:15.948Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:15.809Z InvocationID=5b982fe3-2322-48cd-a402-b4321af9c1d8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Internalvl 2022-04-26T06:22:16.009Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "InternalVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "0f20efcc-cae9-4e21-a80f-bea7530459c1" INVARIANT_UUID = "50e1eee4-94b9-4454-8235-ddf6dfaa5c5e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.internalVL" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0f20efcc-cae9-4e21-a80f-bea7530459c1 EntryTimestamp=2022-04-26T06:22:15.809Z InvocationID=5b982fe3-2322-48cd-a402-b4321af9c1d8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:22:16.010Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "InternalVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "0f20efcc-cae9-4e21-a80f-bea7530459c1" INVARIANT_UUID = "50e1eee4-94b9-4454-8235-ddf6dfaa5c5e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.internalVL" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0f20efcc-cae9-4e21-a80f-bea7530459c1 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:15.809Z InvocationID=5b982fe3-2322-48cd-a402-b4321af9c1d8 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:22:16.054Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "InternalVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "0f20efcc-cae9-4e21-a80f-bea7530459c1" INVARIANT_UUID = "50e1eee4-94b9-4454-8235-ddf6dfaa5c5e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.internalVL" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0f20efcc-cae9-4e21-a80f-bea7530459c1 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:15.809Z InvocationID=5b982fe3-2322-48cd-a402-b4321af9c1d8 ErrorCode=500 lock resource with id 100029a1-5f5d-4920-a8cc-b28760038c5c 2022-04-26T06:22:16.666Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "InternalVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "0f20efcc-cae9-4e21-a80f-bea7530459c1" INVARIANT_UUID = "50e1eee4-94b9-4454-8235-ddf6dfaa5c5e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "100029a1-5f5d-4920-a8cc-b28760038c5c.assettoscatemplate" CURR_ARTIFACT_UUID = "100029a1-5f5d-4920-a8cc-b28760038c5c.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Internalvl-template.yml,0,100029a1-5f5d-4920-a8cc-b28760038c5c.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=740a11b7-0682-435c-8825-30c2155c81bc TargetServiceName=null EntryTimestamp=2022-04-26T06:22:15.809Z InvocationID=5b982fe3-2322-48cd-a402-b4321af9c1d8 ErrorCode=500 Starting to create Csar for component InternalVL 2022-04-26T06:22:16.813Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c64ab8f2-34fa-484c-8a89-14eec4254728 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:22:16.814Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c64ab8f2-34fa-484c-8a89-14eec4254728 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:22:16.913Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "InternalVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "0f20efcc-cae9-4e21-a80f-bea7530459c1" INVARIANT_UUID = "50e1eee4-94b9-4454-8235-ddf6dfaa5c5e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "100029a1-5f5d-4920-a8cc-b28760038c5c.assettoscatemplate" CURR_ARTIFACT_UUID = "100029a1-5f5d-4920-a8cc-b28760038c5c.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Internalvl-template.yml,0,100029a1-5f5d-4920-a8cc-b28760038c5c.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=740a11b7-0682-435c-8825-30c2155c81bc TargetServiceName=null EntryTimestamp=2022-04-26T06:22:15.809Z InvocationID=5b982fe3-2322-48cd-a402-b4321af9c1d8 ErrorCode=500 Ended create Csar for component InternalVL 2022-04-26T06:22:16.914Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "InternalVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "0f20efcc-cae9-4e21-a80f-bea7530459c1" INVARIANT_UUID = "50e1eee4-94b9-4454-8235-ddf6dfaa5c5e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "100029a1-5f5d-4920-a8cc-b28760038c5c.assettoscatemplate" CURR_ARTIFACT_UUID = "100029a1-5f5d-4920-a8cc-b28760038c5c.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Internalvl-template.yml,0,100029a1-5f5d-4920-a8cc-b28760038c5c.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=740a11b7-0682-435c-8825-30c2155c81bc TargetServiceName=null EntryTimestamp=2022-04-26T06:22:15.809Z InvocationID=5b982fe3-2322-48cd-a402-b4321af9c1d8 ErrorCode=500 lock resource with id 100029a1-5f5d-4920-a8cc-b28760038c5c 2022-04-26T06:22:17.463Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "InternalVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "0f20efcc-cae9-4e21-a80f-bea7530459c1" INVARIANT_UUID = "50e1eee4-94b9-4454-8235-ddf6dfaa5c5e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "100029a1-5f5d-4920-a8cc-b28760038c5c.assettoscacsar" CURR_ARTIFACT_UUID = "100029a1-5f5d-4920-a8cc-b28760038c5c.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Internalvl-csar.csar,0,100029a1-5f5d-4920-a8cc-b28760038c5c.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1654 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0c43cb46-6fd1-4c34-8178-e1e0e880781e TargetServiceName=null EntryTimestamp=2022-04-26T06:22:15.809Z InvocationID=5b982fe3-2322-48cd-a402-b4321af9c1d8 ErrorCode=500 LogTimestamp=2022-04-26T06:22:17.463211Z ResponseDesc=Created Exiting. 2022-04-26T06:22:17.464Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:22:17.481Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1f91bc82-b78c-4caf-918f-2911d7ddebe5 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:17.480967Z InvocationID=377b175b-5d55-4aa2-a275-5de2d13b2c1c LogTimestamp=2022-04-26T06:22:17.481073Z Entering 2022-04-26T06:22:17.481Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1f91bc82-b78c-4caf-918f-2911d7ddebe5 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:17.480967Z InvocationID=377b175b-5d55-4aa2-a275-5de2d13b2c1c LogTimestamp=2022-04-26T06:22:17.481073Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:22:17.663Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:17.482Z InvocationID=80b66acd-36a6-4f1e-97d3-ca353fda5c58 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:17.664Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:17.482Z InvocationID=80b66acd-36a6-4f1e-97d3-ca353fda5c58 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:17.664Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:17.482Z InvocationID=80b66acd-36a6-4f1e-97d3-ca353fda5c58 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:17.664Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:17.482Z InvocationID=80b66acd-36a6-4f1e-97d3-ca353fda5c58 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:17.677Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:17.482Z InvocationID=80b66acd-36a6-4f1e-97d3-ca353fda5c58 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Extcp 2022-04-26T06:22:17.772Z [qtp2085002312-15] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ExtCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "864bde39-4264-4ff0-b8dd-23bdd5d61a11" INVARIANT_UUID = "795acc1b-6a31-4aff-816f-62c1d060f020" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.extCP" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=864bde39-4264-4ff0-b8dd-23bdd5d61a11 EntryTimestamp=2022-04-26T06:22:17.482Z InvocationID=80b66acd-36a6-4f1e-97d3-ca353fda5c58 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:22:17.773Z [qtp2085002312-15] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ExtCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "864bde39-4264-4ff0-b8dd-23bdd5d61a11" INVARIANT_UUID = "795acc1b-6a31-4aff-816f-62c1d060f020" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.extCP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=864bde39-4264-4ff0-b8dd-23bdd5d61a11 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:17.482Z InvocationID=80b66acd-36a6-4f1e-97d3-ca353fda5c58 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:22:17.863Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ExtCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "864bde39-4264-4ff0-b8dd-23bdd5d61a11" INVARIANT_UUID = "795acc1b-6a31-4aff-816f-62c1d060f020" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.extCP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=864bde39-4264-4ff0-b8dd-23bdd5d61a11 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:17.482Z InvocationID=80b66acd-36a6-4f1e-97d3-ca353fda5c58 ErrorCode=500 lock resource with id a9359339-061f-42d4-8b1b-7027258c1053 2022-04-26T06:22:18.467Z [qtp2085002312-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ExtCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "864bde39-4264-4ff0-b8dd-23bdd5d61a11" INVARIANT_UUID = "795acc1b-6a31-4aff-816f-62c1d060f020" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a9359339-061f-42d4-8b1b-7027258c1053.assettoscatemplate" CURR_ARTIFACT_UUID = "a9359339-061f-42d4-8b1b-7027258c1053.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extcp-template.yml,0,a9359339-061f-42d4-8b1b-7027258c1053.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6939cf95-1761-4fa0-befa-00f31880bdeb TargetServiceName=null EntryTimestamp=2022-04-26T06:22:17.482Z InvocationID=80b66acd-36a6-4f1e-97d3-ca353fda5c58 ErrorCode=500 Starting to create Csar for component ExtCP 2022-04-26T06:22:18.629Z [qtp2085002312-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ExtCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "864bde39-4264-4ff0-b8dd-23bdd5d61a11" INVARIANT_UUID = "795acc1b-6a31-4aff-816f-62c1d060f020" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a9359339-061f-42d4-8b1b-7027258c1053.assettoscatemplate" CURR_ARTIFACT_UUID = "a9359339-061f-42d4-8b1b-7027258c1053.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extcp-template.yml,0,a9359339-061f-42d4-8b1b-7027258c1053.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6939cf95-1761-4fa0-befa-00f31880bdeb TargetServiceName=null EntryTimestamp=2022-04-26T06:22:17.482Z InvocationID=80b66acd-36a6-4f1e-97d3-ca353fda5c58 ErrorCode=500 Ended create Csar for component ExtCP 2022-04-26T06:22:18.631Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ExtCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "864bde39-4264-4ff0-b8dd-23bdd5d61a11" INVARIANT_UUID = "795acc1b-6a31-4aff-816f-62c1d060f020" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a9359339-061f-42d4-8b1b-7027258c1053.assettoscatemplate" CURR_ARTIFACT_UUID = "a9359339-061f-42d4-8b1b-7027258c1053.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extcp-template.yml,0,a9359339-061f-42d4-8b1b-7027258c1053.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6939cf95-1761-4fa0-befa-00f31880bdeb TargetServiceName=null EntryTimestamp=2022-04-26T06:22:17.482Z InvocationID=80b66acd-36a6-4f1e-97d3-ca353fda5c58 ErrorCode=500 lock resource with id a9359339-061f-42d4-8b1b-7027258c1053 2022-04-26T06:22:18.806Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4bc5b0d-4be3-4dbe-a1d2-b637c0eee737 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:22:18.806Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4bc5b0d-4be3-4dbe-a1d2-b637c0eee737 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@10713ecf 2022-04-26T06:22:18.806Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4bc5b0d-4be3-4dbe-a1d2-b637c0eee737 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:22:18.806Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4bc5b0d-4be3-4dbe-a1d2-b637c0eee737 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:22:19.196Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ExtCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "864bde39-4264-4ff0-b8dd-23bdd5d61a11" INVARIANT_UUID = "795acc1b-6a31-4aff-816f-62c1d060f020" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a9359339-061f-42d4-8b1b-7027258c1053.assettoscacsar" CURR_ARTIFACT_UUID = "a9359339-061f-42d4-8b1b-7027258c1053.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Extcp-csar.csar,0,a9359339-061f-42d4-8b1b-7027258c1053.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1714 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ab58df7e-a74b-4df1-96e3-b340fc2f2561 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:17.482Z InvocationID=80b66acd-36a6-4f1e-97d3-ca353fda5c58 ErrorCode=500 LogTimestamp=2022-04-26T06:22:19.196440Z ResponseDesc=Created Exiting. 2022-04-26T06:22:19.198Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:22:19.220Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=bf5d6dfa-bc30-4fdb-ba13-a51a5efec2be InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:19.219930Z InvocationID=fa0ea07d-b064-4e82-8e5b-4ab7a8f68475 LogTimestamp=2022-04-26T06:22:19.220009Z Entering 2022-04-26T06:22:19.220Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=bf5d6dfa-bc30-4fdb-ba13-a51a5efec2be InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:19.219930Z InvocationID=fa0ea07d-b064-4e82-8e5b-4ab7a8f68475 LogTimestamp=2022-04-26T06:22:19.220009Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:22:19.346Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:19.220Z InvocationID=9bdc2639-1684-4d4f-a011-bf5b954a9af0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:19.347Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:19.220Z InvocationID=9bdc2639-1684-4d4f-a011-bf5b954a9af0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:19.347Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:19.220Z InvocationID=9bdc2639-1684-4d4f-a011-bf5b954a9af0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:19.347Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:19.220Z InvocationID=9bdc2639-1684-4d4f-a011-bf5b954a9af0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:19.359Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:19.220Z InvocationID=9bdc2639-1684-4d4f-a011-bf5b954a9af0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Vl 2022-04-26T06:22:19.469Z [qtp2085002312-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "3a404fd2-effe-4994-b1c8-d22073e00c35" INVARIANT_UUID = "ac2420c6-6559-4166-b034-aeb1b0d5039d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.VL" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3a404fd2-effe-4994-b1c8-d22073e00c35 EntryTimestamp=2022-04-26T06:22:19.220Z InvocationID=9bdc2639-1684-4d4f-a011-bf5b954a9af0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:22:19.470Z [qtp2085002312-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "3a404fd2-effe-4994-b1c8-d22073e00c35" INVARIANT_UUID = "ac2420c6-6559-4166-b034-aeb1b0d5039d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.VL" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3a404fd2-effe-4994-b1c8-d22073e00c35 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:19.220Z InvocationID=9bdc2639-1684-4d4f-a011-bf5b954a9af0 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:22:19.652Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "3a404fd2-effe-4994-b1c8-d22073e00c35" INVARIANT_UUID = "ac2420c6-6559-4166-b034-aeb1b0d5039d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.VL" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3a404fd2-effe-4994-b1c8-d22073e00c35 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:19.220Z InvocationID=9bdc2639-1684-4d4f-a011-bf5b954a9af0 ErrorCode=500 lock resource with id 006549b2-c892-4b45-912b-7b54d4ef94ab 2022-04-26T06:22:19.821Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e4bc5b0d-4be3-4dbe-a1d2-b637c0eee737 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:22:19.822Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e4bc5b0d-4be3-4dbe-a1d2-b637c0eee737 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:22:20.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:22:20.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:22:20.561Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "3a404fd2-effe-4994-b1c8-d22073e00c35" INVARIANT_UUID = "ac2420c6-6559-4166-b034-aeb1b0d5039d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "006549b2-c892-4b45-912b-7b54d4ef94ab.assettoscatemplate" CURR_ARTIFACT_UUID = "006549b2-c892-4b45-912b-7b54d4ef94ab.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vl-template.yml,0,006549b2-c892-4b45-912b-7b54d4ef94ab.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5e2de3b0-a5eb-4d0a-b2d7-d79aab57f273 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:19.220Z InvocationID=9bdc2639-1684-4d4f-a011-bf5b954a9af0 ErrorCode=500 Starting to create Csar for component VL 2022-04-26T06:22:20.795Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "3a404fd2-effe-4994-b1c8-d22073e00c35" INVARIANT_UUID = "ac2420c6-6559-4166-b034-aeb1b0d5039d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "006549b2-c892-4b45-912b-7b54d4ef94ab.assettoscatemplate" CURR_ARTIFACT_UUID = "006549b2-c892-4b45-912b-7b54d4ef94ab.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vl-template.yml,0,006549b2-c892-4b45-912b-7b54d4ef94ab.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5e2de3b0-a5eb-4d0a-b2d7-d79aab57f273 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:19.220Z InvocationID=9bdc2639-1684-4d4f-a011-bf5b954a9af0 ErrorCode=500 Ended create Csar for component VL 2022-04-26T06:22:20.796Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "3a404fd2-effe-4994-b1c8-d22073e00c35" INVARIANT_UUID = "ac2420c6-6559-4166-b034-aeb1b0d5039d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "006549b2-c892-4b45-912b-7b54d4ef94ab.assettoscatemplate" CURR_ARTIFACT_UUID = "006549b2-c892-4b45-912b-7b54d4ef94ab.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vl-template.yml,0,006549b2-c892-4b45-912b-7b54d4ef94ab.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5e2de3b0-a5eb-4d0a-b2d7-d79aab57f273 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:19.220Z InvocationID=9bdc2639-1684-4d4f-a011-bf5b954a9af0 ErrorCode=500 lock resource with id 006549b2-c892-4b45-912b-7b54d4ef94ab 2022-04-26T06:22:21.653Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "3a404fd2-effe-4994-b1c8-d22073e00c35" INVARIANT_UUID = "ac2420c6-6559-4166-b034-aeb1b0d5039d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "006549b2-c892-4b45-912b-7b54d4ef94ab.assettoscacsar" CURR_ARTIFACT_UUID = "006549b2-c892-4b45-912b-7b54d4ef94ab.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Vl-csar.csar,0,006549b2-c892-4b45-912b-7b54d4ef94ab.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=2433 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c6918b27-babc-48ef-abd2-313aa4fcc5ad TargetServiceName=null EntryTimestamp=2022-04-26T06:22:19.220Z InvocationID=9bdc2639-1684-4d4f-a011-bf5b954a9af0 ErrorCode=500 LogTimestamp=2022-04-26T06:22:21.653112Z ResponseDesc=Created Exiting. 2022-04-26T06:22:21.654Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:22:21.745Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=dfca790e-d59d-4de9-8ed0-aa90be169127 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:21.744972Z InvocationID=40fac9d0-dfc9-484d-92be-62df6388590f LogTimestamp=2022-04-26T06:22:21.745061Z Entering 2022-04-26T06:22:21.745Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=dfca790e-d59d-4de9-8ed0-aa90be169127 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:21.744972Z InvocationID=40fac9d0-dfc9-484d-92be-62df6388590f LogTimestamp=2022-04-26T06:22:21.745061Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:22:22.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c55d9507-3b81-4664-bb58-980f36aef63a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:22:22.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c55d9507-3b81-4664-bb58-980f36aef63a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@44c35a0 2022-04-26T06:22:22.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c55d9507-3b81-4664-bb58-980f36aef63a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:22:22.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c55d9507-3b81-4664-bb58-980f36aef63a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:22:22.753Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:21.746Z InvocationID=f5000553-d9e3-4f68-a9c3-05705ccbecb4 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:22.754Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:21.746Z InvocationID=f5000553-d9e3-4f68-a9c3-05705ccbecb4 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:22.754Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:21.746Z InvocationID=f5000553-d9e3-4f68-a9c3-05705ccbecb4 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:22.754Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:21.746Z InvocationID=f5000553-d9e3-4f68-a9c3-05705ccbecb4 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:22.843Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:21.746Z InvocationID=f5000553-d9e3-4f68-a9c3-05705ccbecb4 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name VlEline 2022-04-26T06:22:23.149Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VL ELINE" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "7ade9b2b-46f1-4648-b11a-d4b2491f6317" INVARIANT_UUID = "b9184556-41d4-4a13-8985-66e9a4913e57" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.ELine" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7ade9b2b-46f1-4648-b11a-d4b2491f6317 EntryTimestamp=2022-04-26T06:22:21.746Z InvocationID=f5000553-d9e3-4f68-a9c3-05705ccbecb4 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:22:23.150Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VL ELINE" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "7ade9b2b-46f1-4648-b11a-d4b2491f6317" INVARIANT_UUID = "b9184556-41d4-4a13-8985-66e9a4913e57" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.ELine" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7ade9b2b-46f1-4648-b11a-d4b2491f6317 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:21.746Z InvocationID=f5000553-d9e3-4f68-a9c3-05705ccbecb4 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:22:23.360Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VL ELINE" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "7ade9b2b-46f1-4648-b11a-d4b2491f6317" INVARIANT_UUID = "b9184556-41d4-4a13-8985-66e9a4913e57" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.ELine" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7ade9b2b-46f1-4648-b11a-d4b2491f6317 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:21.746Z InvocationID=f5000553-d9e3-4f68-a9c3-05705ccbecb4 ErrorCode=500 lock resource with id 25883877-4ffe-4285-b140-00ea82735502 2022-04-26T06:22:23.693Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c55d9507-3b81-4664-bb58-980f36aef63a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:22:23.693Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c55d9507-3b81-4664-bb58-980f36aef63a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:22:24.745Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL ELINE" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "7ade9b2b-46f1-4648-b11a-d4b2491f6317" INVARIANT_UUID = "b9184556-41d4-4a13-8985-66e9a4913e57" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "25883877-4ffe-4285-b140-00ea82735502.assettoscatemplate" CURR_ARTIFACT_UUID = "25883877-4ffe-4285-b140-00ea82735502.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VlEline-template.yml,0,25883877-4ffe-4285-b140-00ea82735502.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=af041e30-bc5d-4f14-a611-00a20629ccf5 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:21.746Z InvocationID=f5000553-d9e3-4f68-a9c3-05705ccbecb4 ErrorCode=500 Starting to create Csar for component VL ELINE 2022-04-26T06:22:24.998Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL ELINE" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "7ade9b2b-46f1-4648-b11a-d4b2491f6317" INVARIANT_UUID = "b9184556-41d4-4a13-8985-66e9a4913e57" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "25883877-4ffe-4285-b140-00ea82735502.assettoscatemplate" CURR_ARTIFACT_UUID = "25883877-4ffe-4285-b140-00ea82735502.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VlEline-template.yml,0,25883877-4ffe-4285-b140-00ea82735502.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=af041e30-bc5d-4f14-a611-00a20629ccf5 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:21.746Z InvocationID=f5000553-d9e3-4f68-a9c3-05705ccbecb4 ErrorCode=500 Ended create Csar for component VL ELINE 2022-04-26T06:22:25.001Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL ELINE" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "7ade9b2b-46f1-4648-b11a-d4b2491f6317" INVARIANT_UUID = "b9184556-41d4-4a13-8985-66e9a4913e57" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "25883877-4ffe-4285-b140-00ea82735502.assettoscatemplate" CURR_ARTIFACT_UUID = "25883877-4ffe-4285-b140-00ea82735502.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VlEline-template.yml,0,25883877-4ffe-4285-b140-00ea82735502.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=af041e30-bc5d-4f14-a611-00a20629ccf5 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:21.746Z InvocationID=f5000553-d9e3-4f68-a9c3-05705ccbecb4 ErrorCode=500 lock resource with id 25883877-4ffe-4285-b140-00ea82735502 2022-04-26T06:22:25.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7dcc7436-db1d-4be5-bf3f-fab07d425566 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:22:25.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7dcc7436-db1d-4be5-bf3f-fab07d425566 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@475eb4ff 2022-04-26T06:22:25.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7dcc7436-db1d-4be5-bf3f-fab07d425566 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:22:25.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7dcc7436-db1d-4be5-bf3f-fab07d425566 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:22:25.855Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL ELINE" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "7ade9b2b-46f1-4648-b11a-d4b2491f6317" INVARIANT_UUID = "b9184556-41d4-4a13-8985-66e9a4913e57" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "25883877-4ffe-4285-b140-00ea82735502.assettoscacsar" CURR_ARTIFACT_UUID = "25883877-4ffe-4285-b140-00ea82735502.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VlEline-csar.csar,0,25883877-4ffe-4285-b140-00ea82735502.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=4109 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8ea0f637-cec6-4d1d-8d6b-02f620bce46a TargetServiceName=null EntryTimestamp=2022-04-26T06:22:21.746Z InvocationID=f5000553-d9e3-4f68-a9c3-05705ccbecb4 ErrorCode=500 LogTimestamp=2022-04-26T06:22:25.855034Z ResponseDesc=Created Exiting. 2022-04-26T06:22:25.855Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:22:25.870Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0da78467-875a-45aa-8959-fab7b325dbfc InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:25.869761Z InvocationID=5e99df45-f530-4d0e-a983-2c9a26924aa1 LogTimestamp=2022-04-26T06:22:25.869851Z Entering 2022-04-26T06:22:25.870Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0da78467-875a-45aa-8959-fab7b325dbfc InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:25.869761Z InvocationID=5e99df45-f530-4d0e-a983-2c9a26924aa1 LogTimestamp=2022-04-26T06:22:25.869851Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:22:26.353Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:25.870Z InvocationID=0179366d-c20f-46b0-9004-bcfd2b20f266 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:26.354Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:25.870Z InvocationID=0179366d-c20f-46b0-9004-bcfd2b20f266 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:26.354Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:25.870Z InvocationID=0179366d-c20f-46b0-9004-bcfd2b20f266 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:26.354Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:25.870Z InvocationID=0179366d-c20f-46b0-9004-bcfd2b20f266 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:26.354Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:25.870Z InvocationID=0179366d-c20f-46b0-9004-bcfd2b20f266 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Abstractsubstitute 2022-04-26T06:22:26.672Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=7dcc7436-db1d-4be5-bf3f-fab07d425566 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:22:26.672Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=7dcc7436-db1d-4be5-bf3f-fab07d425566 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:22:26.761Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "AbstractSubstitute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "4976e95d-bf18-47c2-9b8a-0ffc2efa0d61" INVARIANT_UUID = "39fefe3b-419f-4c2c-ac20-c26bd971d74f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.abstract.nodes.AbstractSubstitute" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4976e95d-bf18-47c2-9b8a-0ffc2efa0d61 EntryTimestamp=2022-04-26T06:22:25.870Z InvocationID=0179366d-c20f-46b0-9004-bcfd2b20f266 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id e8f10d0e-b6b3-4df5-9984-a7b64184f021 2022-04-26T06:22:28.456Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "AbstractSubstitute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "4976e95d-bf18-47c2-9b8a-0ffc2efa0d61" INVARIANT_UUID = "39fefe3b-419f-4c2c-ac20-c26bd971d74f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "e8f10d0e-b6b3-4df5-9984-a7b64184f021.assettoscatemplate" CURR_ARTIFACT_UUID = "e8f10d0e-b6b3-4df5-9984-a7b64184f021.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Abstractsubstitute-template.yml,0,e8f10d0e-b6b3-4df5-9984-a7b64184f021.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=2585 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3f2dcab6-f937-46fb-ad62-6f1230d9afe1 EntryTimestamp=2022-04-26T06:22:25.870Z InvocationID=0179366d-c20f-46b0-9004-bcfd2b20f266 LogTimestamp=2022-04-26T06:22:28.455720Z ResponseDesc=Created Exiting. 2022-04-26T06:22:28.456Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:22:28.546Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=782ca392-776c-4cf8-941b-915366eee842 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:28.545644Z InvocationID=528cf71f-1a81-40ba-89df-6eeb5d081927 LogTimestamp=2022-04-26T06:22:28.545801Z Entering 2022-04-26T06:22:28.546Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=782ca392-776c-4cf8-941b-915366eee842 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:28.545644Z InvocationID=528cf71f-1a81-40ba-89df-6eeb5d081927 LogTimestamp=2022-04-26T06:22:28.545801Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:22:28.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ddef62a-9d72-43b7-8d51-8eeffcddd4ec PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:22:28.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ddef62a-9d72-43b7-8d51-8eeffcddd4ec PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6cdb1ec6 2022-04-26T06:22:28.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ddef62a-9d72-43b7-8d51-8eeffcddd4ec PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:22:28.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ddef62a-9d72-43b7-8d51-8eeffcddd4ec PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:22:29.776Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=9ddef62a-9d72-43b7-8d51-8eeffcddd4ec ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:22:29.777Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=9ddef62a-9d72-43b7-8d51-8eeffcddd4ec ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:22:30.558Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:28.546Z InvocationID=f6d478e3-96cd-46c1-acc8-506467f0204f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:30.558Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:28.546Z InvocationID=f6d478e3-96cd-46c1-acc8-506467f0204f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:30.558Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:28.546Z InvocationID=f6d478e3-96cd-46c1-acc8-506467f0204f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:30.558Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:28.546Z InvocationID=f6d478e3-96cd-46c1-acc8-506467f0204f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:30.559Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:28.546Z InvocationID=f6d478e3-96cd-46c1-acc8-506467f0204f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name GenericVfc 2022-04-26T06:22:31.533Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68e60ce7-124e-4890-b09e-1f16417538c1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:22:31.533Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68e60ce7-124e-4890-b09e-1f16417538c1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3781be8 2022-04-26T06:22:31.533Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68e60ce7-124e-4890-b09e-1f16417538c1 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:22:31.533Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68e60ce7-124e-4890-b09e-1f16417538c1 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:22:32.044Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Generic_VFC" RESOURCE_TYPE = "ABSTRACT" SERVICE_INSTANCE_ID = "83da7833-9231-4d2a-8e9b-eac1a6324fd3" INVARIANT_UUID = "b6db62ec-a58d-47f1-a478-e3b80e377b84" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.abstract.nodes.VFC" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=83da7833-9231-4d2a-8e9b-eac1a6324fd3 EntryTimestamp=2022-04-26T06:22:28.546Z InvocationID=f6d478e3-96cd-46c1-acc8-506467f0204f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 51c88b4b-8a9d-4d91-b046-d35b773f25ef 2022-04-26T06:22:32.556Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=68e60ce7-124e-4890-b09e-1f16417538c1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:22:32.557Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=68e60ce7-124e-4890-b09e-1f16417538c1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:22:34.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77a895cd-c79b-4930-87b7-a94af3a5eca2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:22:34.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77a895cd-c79b-4930-87b7-a94af3a5eca2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4012090d 2022-04-26T06:22:34.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77a895cd-c79b-4930-87b7-a94af3a5eca2 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:22:34.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77a895cd-c79b-4930-87b7-a94af3a5eca2 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:22:34.941Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Generic_VFC" RESOURCE_TYPE = "ABSTRACT" SERVICE_INSTANCE_ID = "83da7833-9231-4d2a-8e9b-eac1a6324fd3" INVARIANT_UUID = "b6db62ec-a58d-47f1-a478-e3b80e377b84" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "51c88b4b-8a9d-4d91-b046-d35b773f25ef.assettoscatemplate" CURR_ARTIFACT_UUID = "51c88b4b-8a9d-4d91-b046-d35b773f25ef.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-GenericVfc-template.yml,0,51c88b4b-8a9d-4d91-b046-d35b773f25ef.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=6394 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=db178874-ed62-4fec-87a3-dc0ae070a5c3 EntryTimestamp=2022-04-26T06:22:28.546Z InvocationID=f6d478e3-96cd-46c1-acc8-506467f0204f LogTimestamp=2022-04-26T06:22:34.940974Z ResponseDesc=Created Exiting. 2022-04-26T06:22:34.942Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:22:35.033Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0d0c6a01-e02c-4701-8539-f22337fdbff9 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:35.033371Z InvocationID=e0d80ae8-a1c9-44a3-86b8-642214d83562 LogTimestamp=2022-04-26T06:22:35.033456Z Entering 2022-04-26T06:22:35.034Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0d0c6a01-e02c-4701-8539-f22337fdbff9 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:35.033371Z InvocationID=e0d80ae8-a1c9-44a3-86b8-642214d83562 LogTimestamp=2022-04-26T06:22:35.033456Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:22:35.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:22:35.380Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:22:35.761Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=77a895cd-c79b-4930-87b7-a94af3a5eca2 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:22:35.761Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=77a895cd-c79b-4930-87b7-a94af3a5eca2 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:22:35.847Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:35.034Z InvocationID=a4ee672f-f10d-4f25-bd97-c1489cf89d33 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:35.847Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:35.034Z InvocationID=a4ee672f-f10d-4f25-bd97-c1489cf89d33 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:35.847Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:35.034Z InvocationID=a4ee672f-f10d-4f25-bd97-c1489cf89d33 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:35.847Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:35.034Z InvocationID=a4ee672f-f10d-4f25-bd97-c1489cf89d33 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:35.847Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:35.034Z InvocationID=a4ee672f-f10d-4f25-bd97-c1489cf89d33 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name GenericVf 2022-04-26T06:22:36.656Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Generic_VF" RESOURCE_TYPE = "ABSTRACT" SERVICE_INSTANCE_ID = "50787790-8dd7-481c-b718-48657c39c5f3" INVARIANT_UUID = "bc26d45b-da86-482d-b699-bc46e30f2c61" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.abstract.nodes.VF" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=50787790-8dd7-481c-b718-48657c39c5f3 EntryTimestamp=2022-04-26T06:22:35.034Z InvocationID=a4ee672f-f10d-4f25-bd97-c1489cf89d33 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 704fc15c-1505-460b-a465-bdfae11a3a7d 2022-04-26T06:22:37.639Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=727d431e-bd96-48e2-aa00-aca01acf781d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:22:37.639Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=727d431e-bd96-48e2-aa00-aca01acf781d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f2ad018 2022-04-26T06:22:37.639Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=727d431e-bd96-48e2-aa00-aca01acf781d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:22:37.639Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=727d431e-bd96-48e2-aa00-aca01acf781d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:22:38.357Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Generic_VF" RESOURCE_TYPE = "ABSTRACT" SERVICE_INSTANCE_ID = "50787790-8dd7-481c-b718-48657c39c5f3" INVARIANT_UUID = "bc26d45b-da86-482d-b699-bc46e30f2c61" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "704fc15c-1505-460b-a465-bdfae11a3a7d.assettoscatemplate" CURR_ARTIFACT_UUID = "704fc15c-1505-460b-a465-bdfae11a3a7d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-GenericVf-template.yml,0,704fc15c-1505-460b-a465-bdfae11a3a7d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=3322 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=996843cc-c8a0-4a1d-947b-dcb3bde314ba EntryTimestamp=2022-04-26T06:22:35.034Z InvocationID=a4ee672f-f10d-4f25-bd97-c1489cf89d33 LogTimestamp=2022-04-26T06:22:38.356891Z ResponseDesc=Created Exiting. 2022-04-26T06:22:38.357Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:22:38.441Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1122d396-0f30-43d0-b165-93fcf8803bf6 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:38.440516Z InvocationID=f1a08940-f002-46a3-886e-c41a8314f180 LogTimestamp=2022-04-26T06:22:38.440653Z Entering 2022-04-26T06:22:38.441Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1122d396-0f30-43d0-b165-93fcf8803bf6 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:38.440516Z InvocationID=f1a08940-f002-46a3-886e-c41a8314f180 LogTimestamp=2022-04-26T06:22:38.440653Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:22:38.669Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=727d431e-bd96-48e2-aa00-aca01acf781d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:22:38.670Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=727d431e-bd96-48e2-aa00-aca01acf781d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:22:39.129Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:38.441Z InvocationID=4a0c85a2-971d-48b5-b2d2-5d4fe144e9ae PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:39.130Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:38.441Z InvocationID=4a0c85a2-971d-48b5-b2d2-5d4fe144e9ae PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:39.130Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:38.441Z InvocationID=4a0c85a2-971d-48b5-b2d2-5d4fe144e9ae PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:39.130Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:38.441Z InvocationID=4a0c85a2-971d-48b5-b2d2-5d4fe144e9ae PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:39.131Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:38.441Z InvocationID=4a0c85a2-971d-48b5-b2d2-5d4fe144e9ae PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name GenericCr 2022-04-26T06:22:39.939Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Generic_CR" RESOURCE_TYPE = "ABSTRACT" SERVICE_INSTANCE_ID = "b19b7f17-c8fd-4c3b-b75d-f24e2fff9c10" INVARIANT_UUID = "95880c92-3fe1-4a0e-9049-7f4d0578627e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.abstract.nodes.CR" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b19b7f17-c8fd-4c3b-b75d-f24e2fff9c10 EntryTimestamp=2022-04-26T06:22:38.441Z InvocationID=4a0c85a2-971d-48b5-b2d2-5d4fe144e9ae PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 03b5c871-42a3-45fa-adb6-9d2e79f5b6b3 2022-04-26T06:22:40.447Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0baf973c-7cec-4c44-b7ec-aaa205d04688 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:22:40.447Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0baf973c-7cec-4c44-b7ec-aaa205d04688 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@53a4e2b4 2022-04-26T06:22:40.447Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0baf973c-7cec-4c44-b7ec-aaa205d04688 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:22:40.448Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0baf973c-7cec-4c44-b7ec-aaa205d04688 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:22:40.901Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Generic_CR" RESOURCE_TYPE = "ABSTRACT" SERVICE_INSTANCE_ID = "b19b7f17-c8fd-4c3b-b75d-f24e2fff9c10" INVARIANT_UUID = "95880c92-3fe1-4a0e-9049-7f4d0578627e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "03b5c871-42a3-45fa-adb6-9d2e79f5b6b3.assettoscatemplate" CURR_ARTIFACT_UUID = "03b5c871-42a3-45fa-adb6-9d2e79f5b6b3.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-GenericCr-template.yml,0,03b5c871-42a3-45fa-adb6-9d2e79f5b6b3.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=2459 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3c06fee9-1f1f-42db-832f-1f16cf048809 EntryTimestamp=2022-04-26T06:22:38.441Z InvocationID=4a0c85a2-971d-48b5-b2d2-5d4fe144e9ae LogTimestamp=2022-04-26T06:22:40.900634Z ResponseDesc=Created Exiting. 2022-04-26T06:22:40.902Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:22:40.930Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f5a593ae-9ee3-44dc-949e-ffdc04ec0821 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:40.930237Z InvocationID=07f35872-03da-4c56-9180-1e9bf18721db LogTimestamp=2022-04-26T06:22:40.930374Z Entering 2022-04-26T06:22:40.931Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f5a593ae-9ee3-44dc-949e-ffdc04ec0821 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:40.930237Z InvocationID=07f35872-03da-4c56-9180-1e9bf18721db LogTimestamp=2022-04-26T06:22:40.930374Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:22:41.096Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:40.931Z InvocationID=09e87d7c-b74e-4a48-914d-23e398b94455 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:41.096Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:40.931Z InvocationID=09e87d7c-b74e-4a48-914d-23e398b94455 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:41.096Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:40.931Z InvocationID=09e87d7c-b74e-4a48-914d-23e398b94455 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:41.096Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:40.931Z InvocationID=09e87d7c-b74e-4a48-914d-23e398b94455 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:41.097Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:40.931Z InvocationID=09e87d7c-b74e-4a48-914d-23e398b94455 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name GenericPnf 2022-04-26T06:22:41.180Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Generic_PNF" RESOURCE_TYPE = "ABSTRACT" SERVICE_INSTANCE_ID = "63cd3737-70b9-48e9-9be5-b0185da423c9" INVARIANT_UUID = "dd151312-3a83-425a-9f93-dd0d36d6f9ee" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.abstract.nodes.PNF" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=63cd3737-70b9-48e9-9be5-b0185da423c9 EntryTimestamp=2022-04-26T06:22:40.931Z InvocationID=09e87d7c-b74e-4a48-914d-23e398b94455 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id a1cc5fb5-36cf-4cbb-af91-92084ef8813e 2022-04-26T06:22:41.485Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=0baf973c-7cec-4c44-b7ec-aaa205d04688 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:22:41.485Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=0baf973c-7cec-4c44-b7ec-aaa205d04688 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:22:41.791Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Generic_PNF" RESOURCE_TYPE = "ABSTRACT" SERVICE_INSTANCE_ID = "63cd3737-70b9-48e9-9be5-b0185da423c9" INVARIANT_UUID = "dd151312-3a83-425a-9f93-dd0d36d6f9ee" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a1cc5fb5-36cf-4cbb-af91-92084ef8813e.assettoscatemplate" CURR_ARTIFACT_UUID = "a1cc5fb5-36cf-4cbb-af91-92084ef8813e.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-GenericPnf-template.yml,0,a1cc5fb5-36cf-4cbb-af91-92084ef8813e.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=859 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=735a0445-67dd-477e-9ed2-6ca85713c9a8 EntryTimestamp=2022-04-26T06:22:40.931Z InvocationID=09e87d7c-b74e-4a48-914d-23e398b94455 LogTimestamp=2022-04-26T06:22:41.790869Z ResponseDesc=Created Exiting. 2022-04-26T06:22:41.791Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:22:41.801Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a77cd2dd-863f-4018-9a54-df79cd057140 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:41.801262Z InvocationID=750c1a3e-1af7-40d3-b12b-203726bd71fb LogTimestamp=2022-04-26T06:22:41.801342Z Entering 2022-04-26T06:22:41.801Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a77cd2dd-863f-4018-9a54-df79cd057140 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:41.801262Z InvocationID=750c1a3e-1af7-40d3-b12b-203726bd71fb LogTimestamp=2022-04-26T06:22:41.801342Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:22:41.911Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:41.802Z InvocationID=7b8a40b6-11de-45d4-8798-7c94a3aa38c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:41.911Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:41.802Z InvocationID=7b8a40b6-11de-45d4-8798-7c94a3aa38c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:41.911Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:41.802Z InvocationID=7b8a40b6-11de-45d4-8798-7c94a3aa38c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:41.911Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:41.802Z InvocationID=7b8a40b6-11de-45d4-8798-7c94a3aa38c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:41.911Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:41.802Z InvocationID=7b8a40b6-11de-45d4-8798-7c94a3aa38c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name GenericService 2022-04-26T06:22:41.981Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Generic_Service" RESOURCE_TYPE = "ABSTRACT" SERVICE_INSTANCE_ID = "25aa2e9f-1a99-47f7-8bcc-9499567f17f1" INVARIANT_UUID = "660940c4-03d3-40dc-a46b-62964c201b3c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.abstract.nodes.service" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=25aa2e9f-1a99-47f7-8bcc-9499567f17f1 EntryTimestamp=2022-04-26T06:22:41.802Z InvocationID=7b8a40b6-11de-45d4-8798-7c94a3aa38c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 49535e32-713c-44cd-8da1-c1419954e0c6 2022-04-26T06:22:42.592Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Generic_Service" RESOURCE_TYPE = "ABSTRACT" SERVICE_INSTANCE_ID = "25aa2e9f-1a99-47f7-8bcc-9499567f17f1" INVARIANT_UUID = "660940c4-03d3-40dc-a46b-62964c201b3c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "49535e32-713c-44cd-8da1-c1419954e0c6.assettoscatemplate" CURR_ARTIFACT_UUID = "49535e32-713c-44cd-8da1-c1419954e0c6.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-GenericService-template.yml,0,49535e32-713c-44cd-8da1-c1419954e0c6.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=789 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3feffbc6-3379-42a7-ab90-7fcca8f57714 EntryTimestamp=2022-04-26T06:22:41.802Z InvocationID=7b8a40b6-11de-45d4-8798-7c94a3aa38c5 LogTimestamp=2022-04-26T06:22:42.591745Z ResponseDesc=Created Exiting. 2022-04-26T06:22:42.592Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:22:42.605Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=fd9afd5d-543f-4d29-95f4-83a6453d5dac InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:42.605271Z InvocationID=b889426b-471b-4e9d-928d-17145a336683 LogTimestamp=2022-04-26T06:22:42.605366Z Entering 2022-04-26T06:22:42.606Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=fd9afd5d-543f-4d29-95f4-83a6453d5dac InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:42.605271Z InvocationID=b889426b-471b-4e9d-928d-17145a336683 LogTimestamp=2022-04-26T06:22:42.605366Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:22:42.784Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:42.606Z InvocationID=acea3d4d-44ef-42bb-8d59-2b890a21b0b2 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:42.784Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:42.606Z InvocationID=acea3d4d-44ef-42bb-8d59-2b890a21b0b2 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:42.784Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:42.606Z InvocationID=acea3d4d-44ef-42bb-8d59-2b890a21b0b2 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:42.784Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:42.606Z InvocationID=acea3d4d-44ef-42bb-8d59-2b890a21b0b2 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:42.787Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:42.606Z InvocationID=acea3d4d-44ef-42bb-8d59-2b890a21b0b2 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Contrailnetworkrules 2022-04-26T06:22:42.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59582699-633e-43f9-9280-a2630bd7e5b3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:22:42.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59582699-633e-43f9-9280-a2630bd7e5b3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@372dad94 2022-04-26T06:22:42.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59582699-633e-43f9-9280-a2630bd7e5b3 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:22:42.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59582699-633e-43f9-9280-a2630bd7e5b3 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:22:42.928Z [qtp2085002312-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailNetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e6dbe7ca-4149-408e-8626-c1eb5aff2b13" INVARIANT_UUID = "e5643f2d-1a7f-4fe8-8e33-8d08e038fd17" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.rules.nodes.heat.network.contrail.NetworkRules" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e6dbe7ca-4149-408e-8626-c1eb5aff2b13 EntryTimestamp=2022-04-26T06:22:42.606Z InvocationID=acea3d4d-44ef-42bb-8d59-2b890a21b0b2 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:22:42.928Z [qtp2085002312-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailNetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e6dbe7ca-4149-408e-8626-c1eb5aff2b13" INVARIANT_UUID = "e5643f2d-1a7f-4fe8-8e33-8d08e038fd17" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.rules.nodes.heat.network.contrail.NetworkRules" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e6dbe7ca-4149-408e-8626-c1eb5aff2b13 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:42.606Z InvocationID=acea3d4d-44ef-42bb-8d59-2b890a21b0b2 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:22:42.961Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailNetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e6dbe7ca-4149-408e-8626-c1eb5aff2b13" INVARIANT_UUID = "e5643f2d-1a7f-4fe8-8e33-8d08e038fd17" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.rules.nodes.heat.network.contrail.NetworkRules" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e6dbe7ca-4149-408e-8626-c1eb5aff2b13 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:42.606Z InvocationID=acea3d4d-44ef-42bb-8d59-2b890a21b0b2 ErrorCode=500 lock resource with id af4aa6f1-d669-4e06-8994-e8d3e83f59cc 2022-04-26T06:22:43.629Z [qtp2085002312-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailNetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e6dbe7ca-4149-408e-8626-c1eb5aff2b13" INVARIANT_UUID = "e5643f2d-1a7f-4fe8-8e33-8d08e038fd17" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "af4aa6f1-d669-4e06-8994-e8d3e83f59cc.assettoscatemplate" CURR_ARTIFACT_UUID = "af4aa6f1-d669-4e06-8994-e8d3e83f59cc.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailnetworkrules-template.yml,0,af4aa6f1-d669-4e06-8994-e8d3e83f59cc.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9be7ac4e-5411-4548-a7c6-a1fc06d9e6d3 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:42.606Z InvocationID=acea3d4d-44ef-42bb-8d59-2b890a21b0b2 ErrorCode=500 Starting to create Csar for component ContrailNetworkRules 2022-04-26T06:22:43.809Z [qtp2085002312-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailNetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e6dbe7ca-4149-408e-8626-c1eb5aff2b13" INVARIANT_UUID = "e5643f2d-1a7f-4fe8-8e33-8d08e038fd17" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "af4aa6f1-d669-4e06-8994-e8d3e83f59cc.assettoscatemplate" CURR_ARTIFACT_UUID = "af4aa6f1-d669-4e06-8994-e8d3e83f59cc.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailnetworkrules-template.yml,0,af4aa6f1-d669-4e06-8994-e8d3e83f59cc.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9be7ac4e-5411-4548-a7c6-a1fc06d9e6d3 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:42.606Z InvocationID=acea3d4d-44ef-42bb-8d59-2b890a21b0b2 ErrorCode=500 Ended create Csar for component ContrailNetworkRules 2022-04-26T06:22:43.812Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailNetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e6dbe7ca-4149-408e-8626-c1eb5aff2b13" INVARIANT_UUID = "e5643f2d-1a7f-4fe8-8e33-8d08e038fd17" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "af4aa6f1-d669-4e06-8994-e8d3e83f59cc.assettoscatemplate" CURR_ARTIFACT_UUID = "af4aa6f1-d669-4e06-8994-e8d3e83f59cc.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailnetworkrules-template.yml,0,af4aa6f1-d669-4e06-8994-e8d3e83f59cc.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9be7ac4e-5411-4548-a7c6-a1fc06d9e6d3 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:42.606Z InvocationID=acea3d4d-44ef-42bb-8d59-2b890a21b0b2 ErrorCode=500 lock resource with id af4aa6f1-d669-4e06-8994-e8d3e83f59cc 2022-04-26T06:22:43.886Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=59582699-633e-43f9-9280-a2630bd7e5b3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:22:43.887Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=59582699-633e-43f9-9280-a2630bd7e5b3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:22:44.372Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailNetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e6dbe7ca-4149-408e-8626-c1eb5aff2b13" INVARIANT_UUID = "e5643f2d-1a7f-4fe8-8e33-8d08e038fd17" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "af4aa6f1-d669-4e06-8994-e8d3e83f59cc.assettoscacsar" CURR_ARTIFACT_UUID = "af4aa6f1-d669-4e06-8994-e8d3e83f59cc.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Contrailnetworkrules-csar.csar,0,af4aa6f1-d669-4e06-8994-e8d3e83f59cc.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1766 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3e550471-ad95-4074-a09c-7c2c93f33aa9 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:42.606Z InvocationID=acea3d4d-44ef-42bb-8d59-2b890a21b0b2 ErrorCode=500 LogTimestamp=2022-04-26T06:22:44.372403Z ResponseDesc=Created Exiting. 2022-04-26T06:22:44.373Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:22:44.386Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=775fb727-306f-4bde-a850-18e1e41f31ba InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:44.386337Z InvocationID=498116a3-a5b0-407c-a094-760256b9a8fd LogTimestamp=2022-04-26T06:22:44.386416Z Entering 2022-04-26T06:22:44.387Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=775fb727-306f-4bde-a850-18e1e41f31ba InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:44.386337Z InvocationID=498116a3-a5b0-407c-a094-760256b9a8fd LogTimestamp=2022-04-26T06:22:44.386416Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:22:44.475Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:44.387Z InvocationID=f1aa90e5-e704-4b54-bd93-ed6a5a8b4663 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:44.475Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:44.387Z InvocationID=f1aa90e5-e704-4b54-bd93-ed6a5a8b4663 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:44.476Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:44.387Z InvocationID=f1aa90e5-e704-4b54-bd93-ed6a5a8b4663 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:44.476Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:44.387Z InvocationID=f1aa90e5-e704-4b54-bd93-ed6a5a8b4663 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:44.476Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:44.387Z InvocationID=f1aa90e5-e704-4b54-bd93-ed6a5a8b4663 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Contrailport 2022-04-26T06:22:44.523Z [qtp2085002312-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "ccc0d52f-4372-4879-bbdc-900fa7561038" INVARIANT_UUID = "2884b909-f97a-47a2-a17e-e36566b4929f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.network.contrail.Port" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ccc0d52f-4372-4879-bbdc-900fa7561038 EntryTimestamp=2022-04-26T06:22:44.387Z InvocationID=f1aa90e5-e704-4b54-bd93-ed6a5a8b4663 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:22:44.524Z [qtp2085002312-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "ccc0d52f-4372-4879-bbdc-900fa7561038" INVARIANT_UUID = "2884b909-f97a-47a2-a17e-e36566b4929f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.network.contrail.Port" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ccc0d52f-4372-4879-bbdc-900fa7561038 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:44.387Z InvocationID=f1aa90e5-e704-4b54-bd93-ed6a5a8b4663 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:22:44.564Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "ccc0d52f-4372-4879-bbdc-900fa7561038" INVARIANT_UUID = "2884b909-f97a-47a2-a17e-e36566b4929f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.network.contrail.Port" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ccc0d52f-4372-4879-bbdc-900fa7561038 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:44.387Z InvocationID=f1aa90e5-e704-4b54-bd93-ed6a5a8b4663 ErrorCode=500 lock resource with id 5e869a4d-1728-46a8-bcb6-4c1d1e001bab 2022-04-26T06:22:45.238Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "ccc0d52f-4372-4879-bbdc-900fa7561038" INVARIANT_UUID = "2884b909-f97a-47a2-a17e-e36566b4929f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5e869a4d-1728-46a8-bcb6-4c1d1e001bab.assettoscatemplate" CURR_ARTIFACT_UUID = "5e869a4d-1728-46a8-bcb6-4c1d1e001bab.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailport-template.yml,0,5e869a4d-1728-46a8-bcb6-4c1d1e001bab.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bc935a9f-b84e-46c2-9413-41c68bbbc74b TargetServiceName=null EntryTimestamp=2022-04-26T06:22:44.387Z InvocationID=f1aa90e5-e704-4b54-bd93-ed6a5a8b4663 ErrorCode=500 Starting to create Csar for component ContrailPort 2022-04-26T06:22:45.404Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "ccc0d52f-4372-4879-bbdc-900fa7561038" INVARIANT_UUID = "2884b909-f97a-47a2-a17e-e36566b4929f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5e869a4d-1728-46a8-bcb6-4c1d1e001bab.assettoscatemplate" CURR_ARTIFACT_UUID = "5e869a4d-1728-46a8-bcb6-4c1d1e001bab.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailport-template.yml,0,5e869a4d-1728-46a8-bcb6-4c1d1e001bab.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bc935a9f-b84e-46c2-9413-41c68bbbc74b TargetServiceName=null EntryTimestamp=2022-04-26T06:22:44.387Z InvocationID=f1aa90e5-e704-4b54-bd93-ed6a5a8b4663 ErrorCode=500 Ended create Csar for component ContrailPort 2022-04-26T06:22:45.405Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "ccc0d52f-4372-4879-bbdc-900fa7561038" INVARIANT_UUID = "2884b909-f97a-47a2-a17e-e36566b4929f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5e869a4d-1728-46a8-bcb6-4c1d1e001bab.assettoscatemplate" CURR_ARTIFACT_UUID = "5e869a4d-1728-46a8-bcb6-4c1d1e001bab.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailport-template.yml,0,5e869a4d-1728-46a8-bcb6-4c1d1e001bab.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bc935a9f-b84e-46c2-9413-41c68bbbc74b TargetServiceName=null EntryTimestamp=2022-04-26T06:22:44.387Z InvocationID=f1aa90e5-e704-4b54-bd93-ed6a5a8b4663 ErrorCode=500 lock resource with id 5e869a4d-1728-46a8-bcb6-4c1d1e001bab 2022-04-26T06:22:45.947Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2488703c-6853-4d64-9056-6e815180003f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:22:45.947Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2488703c-6853-4d64-9056-6e815180003f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@367e024c 2022-04-26T06:22:45.947Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2488703c-6853-4d64-9056-6e815180003f PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:22:45.948Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2488703c-6853-4d64-9056-6e815180003f PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:22:46.037Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "ccc0d52f-4372-4879-bbdc-900fa7561038" INVARIANT_UUID = "2884b909-f97a-47a2-a17e-e36566b4929f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5e869a4d-1728-46a8-bcb6-4c1d1e001bab.assettoscacsar" CURR_ARTIFACT_UUID = "5e869a4d-1728-46a8-bcb6-4c1d1e001bab.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Contrailport-csar.csar,0,5e869a4d-1728-46a8-bcb6-4c1d1e001bab.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1650 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3bf74a1b-a896-4563-85e5-eb6997582b7d TargetServiceName=null EntryTimestamp=2022-04-26T06:22:44.387Z InvocationID=f1aa90e5-e704-4b54-bd93-ed6a5a8b4663 ErrorCode=500 LogTimestamp=2022-04-26T06:22:46.037209Z ResponseDesc=Created Exiting. 2022-04-26T06:22:46.038Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:22:46.051Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=d64a9386-f075-4bbe-9d34-48412b097618 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:46.051457Z InvocationID=741b36bd-587c-41bb-8489-4de5c97ea3f7 LogTimestamp=2022-04-26T06:22:46.051534Z Entering 2022-04-26T06:22:46.052Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=d64a9386-f075-4bbe-9d34-48412b097618 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:46.051457Z InvocationID=741b36bd-587c-41bb-8489-4de5c97ea3f7 LogTimestamp=2022-04-26T06:22:46.051534Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:22:46.140Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:46.052Z InvocationID=9c7c2200-3f4e-4135-96e9-a643528ff985 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:46.140Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:46.052Z InvocationID=9c7c2200-3f4e-4135-96e9-a643528ff985 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:46.141Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:46.052Z InvocationID=9c7c2200-3f4e-4135-96e9-a643528ff985 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:46.141Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:46.052Z InvocationID=9c7c2200-3f4e-4135-96e9-a643528ff985 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:46.150Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:46.052Z InvocationID=9c7c2200-3f4e-4135-96e9-a643528ff985 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name PortMirroringConfiguration 2022-04-26T06:22:46.255Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Port Mirroring Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "56213dee-eaca-4647-b937-9afb3b8b4358" INVARIANT_UUID = "60208188-7346-45e2-aae0-29218ce6b2c5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.nodes.PortMirroringConfiguration" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=56213dee-eaca-4647-b937-9afb3b8b4358 EntryTimestamp=2022-04-26T06:22:46.052Z InvocationID=9c7c2200-3f4e-4135-96e9-a643528ff985 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 3954d6dd-8070-4b48-81cf-cf7872736624 2022-04-26T06:22:46.858Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Port Mirroring Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "56213dee-eaca-4647-b937-9afb3b8b4358" INVARIANT_UUID = "60208188-7346-45e2-aae0-29218ce6b2c5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "3954d6dd-8070-4b48-81cf-cf7872736624.assettoscatemplate" CURR_ARTIFACT_UUID = "3954d6dd-8070-4b48-81cf-cf7872736624.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-PortMirroringConfiguration-template.yml,0,3954d6dd-8070-4b48-81cf-cf7872736624.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=35cfe1fe-a639-4163-9d72-8683f5e3187f EntryTimestamp=2022-04-26T06:22:46.052Z InvocationID=9c7c2200-3f4e-4135-96e9-a643528ff985 PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Starting to create Csar for component Port Mirroring Configuration 2022-04-26T06:22:46.995Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=2488703c-6853-4d64-9056-6e815180003f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:22:46.996Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=2488703c-6853-4d64-9056-6e815180003f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:22:47.144Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Port Mirroring Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "56213dee-eaca-4647-b937-9afb3b8b4358" INVARIANT_UUID = "60208188-7346-45e2-aae0-29218ce6b2c5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "3954d6dd-8070-4b48-81cf-cf7872736624.assettoscatemplate" CURR_ARTIFACT_UUID = "3954d6dd-8070-4b48-81cf-cf7872736624.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-PortMirroringConfiguration-template.yml,0,3954d6dd-8070-4b48-81cf-cf7872736624.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=35cfe1fe-a639-4163-9d72-8683f5e3187f EntryTimestamp=2022-04-26T06:22:46.052Z InvocationID=9c7c2200-3f4e-4135-96e9-a643528ff985 Ended create Csar for component Port Mirroring Configuration 2022-04-26T06:22:47.145Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Port Mirroring Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "56213dee-eaca-4647-b937-9afb3b8b4358" INVARIANT_UUID = "60208188-7346-45e2-aae0-29218ce6b2c5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "3954d6dd-8070-4b48-81cf-cf7872736624.assettoscatemplate" CURR_ARTIFACT_UUID = "3954d6dd-8070-4b48-81cf-cf7872736624.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-PortMirroringConfiguration-template.yml,0,3954d6dd-8070-4b48-81cf-cf7872736624.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=35cfe1fe-a639-4163-9d72-8683f5e3187f EntryTimestamp=2022-04-26T06:22:46.052Z InvocationID=9c7c2200-3f4e-4135-96e9-a643528ff985 lock resource with id 3954d6dd-8070-4b48-81cf-cf7872736624 2022-04-26T06:22:47.710Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Port Mirroring Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "56213dee-eaca-4647-b937-9afb3b8b4358" INVARIANT_UUID = "60208188-7346-45e2-aae0-29218ce6b2c5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "3954d6dd-8070-4b48-81cf-cf7872736624.assettoscacsar" CURR_ARTIFACT_UUID = "3954d6dd-8070-4b48-81cf-cf7872736624.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-PortMirroringConfiguration-csar.csar,0,3954d6dd-8070-4b48-81cf-cf7872736624.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1658 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b106c8ff-7689-4f10-8882-a661d66902a5 EntryTimestamp=2022-04-26T06:22:46.052Z InvocationID=9c7c2200-3f4e-4135-96e9-a643528ff985 LogTimestamp=2022-04-26T06:22:47.710262Z ResponseDesc=Created Exiting. 2022-04-26T06:22:47.711Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:22:47.727Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=4626f2e8-8ea3-4015-9b0f-d6242f3ab78e InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:47.726957Z InvocationID=0646a791-ed74-42c8-8caf-8a49ef49d1b7 LogTimestamp=2022-04-26T06:22:47.727034Z Entering 2022-04-26T06:22:47.727Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=4626f2e8-8ea3-4015-9b0f-d6242f3ab78e InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:47.726957Z InvocationID=0646a791-ed74-42c8-8caf-8a49ef49d1b7 LogTimestamp=2022-04-26T06:22:47.727034Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:22:47.834Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:47.728Z InvocationID=b44b9e19-900f-42d4-8374-abc92b941346 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:47.835Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:47.728Z InvocationID=b44b9e19-900f-42d4-8374-abc92b941346 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:47.835Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:47.728Z InvocationID=b44b9e19-900f-42d4-8374-abc92b941346 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:47.835Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:47.728Z InvocationID=b44b9e19-900f-42d4-8374-abc92b941346 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:47.836Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:47.728Z InvocationID=b44b9e19-900f-42d4-8374-abc92b941346 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Serviceproxy 2022-04-26T06:22:47.909Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "serviceProxy" RESOURCE_TYPE = "ServiceProxy" SERVICE_INSTANCE_ID = "a0751044-200e-442b-8062-48f370166d34" INVARIANT_UUID = "dcbaaf31-4310-4b51-94e8-e3ca9c82d756" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.nodes.ServiceProxy" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a0751044-200e-442b-8062-48f370166d34 EntryTimestamp=2022-04-26T06:22:47.728Z InvocationID=b44b9e19-900f-42d4-8374-abc92b941346 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 85f87334-bfd8-4afe-a7ea-ddd5b6ba3202 2022-04-26T06:22:48.549Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "serviceProxy" RESOURCE_TYPE = "ServiceProxy" SERVICE_INSTANCE_ID = "a0751044-200e-442b-8062-48f370166d34" INVARIANT_UUID = "dcbaaf31-4310-4b51-94e8-e3ca9c82d756" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "85f87334-bfd8-4afe-a7ea-ddd5b6ba3202.assettoscatemplate" CURR_ARTIFACT_UUID = "85f87334-bfd8-4afe-a7ea-ddd5b6ba3202.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Serviceproxy-template.yml,0,85f87334-bfd8-4afe-a7ea-ddd5b6ba3202.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=821 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ae9cb8e6-0753-4cc9-bdbe-ca67e88854e9 EntryTimestamp=2022-04-26T06:22:47.728Z InvocationID=b44b9e19-900f-42d4-8374-abc92b941346 LogTimestamp=2022-04-26T06:22:48.549243Z ResponseDesc=Created Exiting. 2022-04-26T06:22:48.550Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:22:48.560Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=b4033168-1b99-48bb-84ce-4ccb729f8bc1 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:48.560466Z InvocationID=d690401c-22a6-41f5-9e10-b77ddc4ce54c LogTimestamp=2022-04-26T06:22:48.560542Z Entering 2022-04-26T06:22:48.561Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=b4033168-1b99-48bb-84ce-4ccb729f8bc1 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:48.560466Z InvocationID=d690401c-22a6-41f5-9e10-b77ddc4ce54c LogTimestamp=2022-04-26T06:22:48.560542Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:22:48.668Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:48.561Z InvocationID=77445c16-874c-470b-93ab-344944f3dc61 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:48.669Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:48.561Z InvocationID=77445c16-874c-470b-93ab-344944f3dc61 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:48.669Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:48.561Z InvocationID=77445c16-874c-470b-93ab-344944f3dc61 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:48.669Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:48.561Z InvocationID=77445c16-874c-470b-93ab-344944f3dc61 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:48.673Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:48.561Z InvocationID=77445c16-874c-470b-93ab-344944f3dc61 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Contrailv2networkrules 2022-04-26T06:22:48.757Z [qtp2085002312-15] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailV2NetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "f236154c-adfc-4973-ae0d-e34ace8fa999" INVARIANT_UUID = "0166c58d-7e99-4d72-a3c8-6c7ab740f15e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.rules.nodes.heat.network.contrailV2.NetworkRules" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f236154c-adfc-4973-ae0d-e34ace8fa999 EntryTimestamp=2022-04-26T06:22:48.561Z InvocationID=77445c16-874c-470b-93ab-344944f3dc61 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:22:48.758Z [qtp2085002312-15] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailV2NetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "f236154c-adfc-4973-ae0d-e34ace8fa999" INVARIANT_UUID = "0166c58d-7e99-4d72-a3c8-6c7ab740f15e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.rules.nodes.heat.network.contrailV2.NetworkRules" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f236154c-adfc-4973-ae0d-e34ace8fa999 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:48.561Z InvocationID=77445c16-874c-470b-93ab-344944f3dc61 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:22:48.864Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailV2NetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "f236154c-adfc-4973-ae0d-e34ace8fa999" INVARIANT_UUID = "0166c58d-7e99-4d72-a3c8-6c7ab740f15e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.rules.nodes.heat.network.contrailV2.NetworkRules" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f236154c-adfc-4973-ae0d-e34ace8fa999 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:48.561Z InvocationID=77445c16-874c-470b-93ab-344944f3dc61 ErrorCode=500 lock resource with id 031b8510-51e3-4060-9813-c1e6f3a92d55 2022-04-26T06:22:48.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e277c451-a1b8-4625-a884-6b4249a86cbf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:22:48.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e277c451-a1b8-4625-a884-6b4249a86cbf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5257d3d6 2022-04-26T06:22:48.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e277c451-a1b8-4625-a884-6b4249a86cbf PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:22:48.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e277c451-a1b8-4625-a884-6b4249a86cbf PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:22:49.743Z [qtp2085002312-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailV2NetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "f236154c-adfc-4973-ae0d-e34ace8fa999" INVARIANT_UUID = "0166c58d-7e99-4d72-a3c8-6c7ab740f15e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "031b8510-51e3-4060-9813-c1e6f3a92d55.assettoscatemplate" CURR_ARTIFACT_UUID = "031b8510-51e3-4060-9813-c1e6f3a92d55.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2networkrules-template.yml,0,031b8510-51e3-4060-9813-c1e6f3a92d55.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3efe9f44-462f-4dc6-911a-21dd98f7deff TargetServiceName=null EntryTimestamp=2022-04-26T06:22:48.561Z InvocationID=77445c16-874c-470b-93ab-344944f3dc61 ErrorCode=500 Starting to create Csar for component ContrailV2NetworkRules 2022-04-26T06:22:49.914Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e277c451-a1b8-4625-a884-6b4249a86cbf ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:22:49.915Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e277c451-a1b8-4625-a884-6b4249a86cbf ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:22:49.925Z [qtp2085002312-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailV2NetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "f236154c-adfc-4973-ae0d-e34ace8fa999" INVARIANT_UUID = "0166c58d-7e99-4d72-a3c8-6c7ab740f15e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "031b8510-51e3-4060-9813-c1e6f3a92d55.assettoscatemplate" CURR_ARTIFACT_UUID = "031b8510-51e3-4060-9813-c1e6f3a92d55.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2networkrules-template.yml,0,031b8510-51e3-4060-9813-c1e6f3a92d55.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3efe9f44-462f-4dc6-911a-21dd98f7deff TargetServiceName=null EntryTimestamp=2022-04-26T06:22:48.561Z InvocationID=77445c16-874c-470b-93ab-344944f3dc61 ErrorCode=500 Ended create Csar for component ContrailV2NetworkRules 2022-04-26T06:22:49.925Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailV2NetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "f236154c-adfc-4973-ae0d-e34ace8fa999" INVARIANT_UUID = "0166c58d-7e99-4d72-a3c8-6c7ab740f15e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "031b8510-51e3-4060-9813-c1e6f3a92d55.assettoscatemplate" CURR_ARTIFACT_UUID = "031b8510-51e3-4060-9813-c1e6f3a92d55.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2networkrules-template.yml,0,031b8510-51e3-4060-9813-c1e6f3a92d55.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3efe9f44-462f-4dc6-911a-21dd98f7deff TargetServiceName=null EntryTimestamp=2022-04-26T06:22:48.561Z InvocationID=77445c16-874c-470b-93ab-344944f3dc61 ErrorCode=500 lock resource with id 031b8510-51e3-4060-9813-c1e6f3a92d55 2022-04-26T06:22:50.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:22:50.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:22:50.555Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailV2NetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "f236154c-adfc-4973-ae0d-e34ace8fa999" INVARIANT_UUID = "0166c58d-7e99-4d72-a3c8-6c7ab740f15e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "031b8510-51e3-4060-9813-c1e6f3a92d55.assettoscacsar" CURR_ARTIFACT_UUID = "031b8510-51e3-4060-9813-c1e6f3a92d55.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Contrailv2networkrules-csar.csar,0,031b8510-51e3-4060-9813-c1e6f3a92d55.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1994 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0db9bbf6-391a-4bfb-9edf-225081378386 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:48.561Z InvocationID=77445c16-874c-470b-93ab-344944f3dc61 ErrorCode=500 LogTimestamp=2022-04-26T06:22:50.555006Z ResponseDesc=Created Exiting. 2022-04-26T06:22:50.556Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:22:50.644Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=82d2c907-724a-495b-a762-a8087901bf12 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:50.644572Z InvocationID=198ecdaf-4585-41b6-8971-be03979dbe26 LogTimestamp=2022-04-26T06:22:50.644658Z Entering 2022-04-26T06:22:50.645Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=82d2c907-724a-495b-a762-a8087901bf12 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:50.644572Z InvocationID=198ecdaf-4585-41b6-8971-be03979dbe26 LogTimestamp=2022-04-26T06:22:50.644658Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:22:51.261Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:50.645Z InvocationID=13e9868e-939c-4b8c-b345-1d661f45484a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:51.261Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:50.645Z InvocationID=13e9868e-939c-4b8c-b345-1d661f45484a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:51.261Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:50.645Z InvocationID=13e9868e-939c-4b8c-b345-1d661f45484a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:51.262Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:50.645Z InvocationID=13e9868e-939c-4b8c-b345-1d661f45484a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:51.344Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:50.645Z InvocationID=13e9868e-939c-4b8c-b345-1d661f45484a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Contrailv2virtualnetwork 2022-04-26T06:22:51.459Z [qtp2085002312-13] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailV2VirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "a41ef1b5-0b65-4c55-85cf-93ee7b2ee859" INVARIANT_UUID = "fb4a364e-f8fe-4ae2-bdfc-c8736ce3d92e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.nodes.heat.network.contrailV2.VirtualNetwork" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a41ef1b5-0b65-4c55-85cf-93ee7b2ee859 EntryTimestamp=2022-04-26T06:22:50.645Z InvocationID=13e9868e-939c-4b8c-b345-1d661f45484a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:22:51.460Z [qtp2085002312-13] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailV2VirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "a41ef1b5-0b65-4c55-85cf-93ee7b2ee859" INVARIANT_UUID = "fb4a364e-f8fe-4ae2-bdfc-c8736ce3d92e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.nodes.heat.network.contrailV2.VirtualNetwork" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a41ef1b5-0b65-4c55-85cf-93ee7b2ee859 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:50.645Z InvocationID=13e9868e-939c-4b8c-b345-1d661f45484a ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:22:51.654Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailV2VirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "a41ef1b5-0b65-4c55-85cf-93ee7b2ee859" INVARIANT_UUID = "fb4a364e-f8fe-4ae2-bdfc-c8736ce3d92e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.nodes.heat.network.contrailV2.VirtualNetwork" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a41ef1b5-0b65-4c55-85cf-93ee7b2ee859 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:50.645Z InvocationID=13e9868e-939c-4b8c-b345-1d661f45484a ErrorCode=500 lock resource with id 2e4d26aa-9945-4135-964f-a1734ea91074 2022-04-26T06:22:52.548Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee5a9e46-f29d-42dd-abaf-c3c45aa80aa5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:22:52.548Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee5a9e46-f29d-42dd-abaf-c3c45aa80aa5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@994ab29 2022-04-26T06:22:52.548Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee5a9e46-f29d-42dd-abaf-c3c45aa80aa5 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:22:52.548Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee5a9e46-f29d-42dd-abaf-c3c45aa80aa5 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:22:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:22:52.686Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:22:52.687Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:22:52.952Z [qtp2085002312-13] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailV2VirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "a41ef1b5-0b65-4c55-85cf-93ee7b2ee859" INVARIANT_UUID = "fb4a364e-f8fe-4ae2-bdfc-c8736ce3d92e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2e4d26aa-9945-4135-964f-a1734ea91074.assettoscatemplate" CURR_ARTIFACT_UUID = "2e4d26aa-9945-4135-964f-a1734ea91074.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2virtualnetwork-template.yml,0,2e4d26aa-9945-4135-964f-a1734ea91074.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=232589d5-ce2c-4675-a231-45061a6a4d34 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:50.645Z InvocationID=13e9868e-939c-4b8c-b345-1d661f45484a ErrorCode=500 Starting to create Csar for component ContrailV2VirtualNetwork 2022-04-26T06:22:53.113Z [qtp2085002312-13] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailV2VirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "a41ef1b5-0b65-4c55-85cf-93ee7b2ee859" INVARIANT_UUID = "fb4a364e-f8fe-4ae2-bdfc-c8736ce3d92e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2e4d26aa-9945-4135-964f-a1734ea91074.assettoscatemplate" CURR_ARTIFACT_UUID = "2e4d26aa-9945-4135-964f-a1734ea91074.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2virtualnetwork-template.yml,0,2e4d26aa-9945-4135-964f-a1734ea91074.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=232589d5-ce2c-4675-a231-45061a6a4d34 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:50.645Z InvocationID=13e9868e-939c-4b8c-b345-1d661f45484a ErrorCode=500 Ended create Csar for component ContrailV2VirtualNetwork 2022-04-26T06:22:53.114Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailV2VirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "a41ef1b5-0b65-4c55-85cf-93ee7b2ee859" INVARIANT_UUID = "fb4a364e-f8fe-4ae2-bdfc-c8736ce3d92e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2e4d26aa-9945-4135-964f-a1734ea91074.assettoscatemplate" CURR_ARTIFACT_UUID = "2e4d26aa-9945-4135-964f-a1734ea91074.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2virtualnetwork-template.yml,0,2e4d26aa-9945-4135-964f-a1734ea91074.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=232589d5-ce2c-4675-a231-45061a6a4d34 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:50.645Z InvocationID=13e9868e-939c-4b8c-b345-1d661f45484a ErrorCode=500 lock resource with id 2e4d26aa-9945-4135-964f-a1734ea91074 2022-04-26T06:22:53.581Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ee5a9e46-f29d-42dd-abaf-c3c45aa80aa5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:22:53.581Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ee5a9e46-f29d-42dd-abaf-c3c45aa80aa5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:22:53.760Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailV2VirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "a41ef1b5-0b65-4c55-85cf-93ee7b2ee859" INVARIANT_UUID = "fb4a364e-f8fe-4ae2-bdfc-c8736ce3d92e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2e4d26aa-9945-4135-964f-a1734ea91074.assettoscacsar" CURR_ARTIFACT_UUID = "2e4d26aa-9945-4135-964f-a1734ea91074.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Contrailv2virtualnetwork-csar.csar,0,2e4d26aa-9945-4135-964f-a1734ea91074.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=3115 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3fa923ad-f4f2-41ba-a780-bfa83dd38885 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:50.645Z InvocationID=13e9868e-939c-4b8c-b345-1d661f45484a ErrorCode=500 LogTimestamp=2022-04-26T06:22:53.760314Z ResponseDesc=Created Exiting. 2022-04-26T06:22:53.761Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:22:53.846Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6ce53c9a-aaa7-47e3-80c2-1637a821256b InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:53.846214Z InvocationID=dcf5a93b-2805-4a1a-af75-ce0d797b5787 LogTimestamp=2022-04-26T06:22:53.846342Z Entering 2022-04-26T06:22:53.847Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6ce53c9a-aaa7-47e3-80c2-1637a821256b InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:53.846214Z InvocationID=dcf5a93b-2805-4a1a-af75-ce0d797b5787 LogTimestamp=2022-04-26T06:22:53.846342Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:22:54.444Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:53.847Z InvocationID=52641d6b-5f6e-4e83-a3e2-243a0a9ef788 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:54.445Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:53.847Z InvocationID=52641d6b-5f6e-4e83-a3e2-243a0a9ef788 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:54.445Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:53.847Z InvocationID=52641d6b-5f6e-4e83-a3e2-243a0a9ef788 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:54.445Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:53.847Z InvocationID=52641d6b-5f6e-4e83-a3e2-243a0a9ef788 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:54.449Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:53.847Z InvocationID=52641d6b-5f6e-4e83-a3e2-243a0a9ef788 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Securityrules 2022-04-26T06:22:54.742Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "SecurityRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "4016522f-bbcc-4490-8913-6d4e342dec41" INVARIANT_UUID = "7712c948-72b7-4e7e-a04d-69de58b8005c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.rules.nodes.heat.network.neutron.SecurityRules" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4016522f-bbcc-4490-8913-6d4e342dec41 EntryTimestamp=2022-04-26T06:22:53.847Z InvocationID=52641d6b-5f6e-4e83-a3e2-243a0a9ef788 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:22:54.743Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "SecurityRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "4016522f-bbcc-4490-8913-6d4e342dec41" INVARIANT_UUID = "7712c948-72b7-4e7e-a04d-69de58b8005c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.rules.nodes.heat.network.neutron.SecurityRules" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4016522f-bbcc-4490-8913-6d4e342dec41 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:53.847Z InvocationID=52641d6b-5f6e-4e83-a3e2-243a0a9ef788 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:22:55.054Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "SecurityRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "4016522f-bbcc-4490-8913-6d4e342dec41" INVARIANT_UUID = "7712c948-72b7-4e7e-a04d-69de58b8005c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.rules.nodes.heat.network.neutron.SecurityRules" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4016522f-bbcc-4490-8913-6d4e342dec41 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:53.847Z InvocationID=52641d6b-5f6e-4e83-a3e2-243a0a9ef788 ErrorCode=500 lock resource with id dc1a6291-408a-4965-9f94-f73e77448c43 2022-04-26T06:22:55.644Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=951e4b42-15c6-4ead-ab70-fc2172f73657 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:22:55.644Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=951e4b42-15c6-4ead-ab70-fc2172f73657 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9e39cfe 2022-04-26T06:22:55.644Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=951e4b42-15c6-4ead-ab70-fc2172f73657 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:22:55.644Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=951e4b42-15c6-4ead-ab70-fc2172f73657 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:22:56.553Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "SecurityRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "4016522f-bbcc-4490-8913-6d4e342dec41" INVARIANT_UUID = "7712c948-72b7-4e7e-a04d-69de58b8005c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "dc1a6291-408a-4965-9f94-f73e77448c43.assettoscatemplate" CURR_ARTIFACT_UUID = "dc1a6291-408a-4965-9f94-f73e77448c43.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Securityrules-template.yml,0,dc1a6291-408a-4965-9f94-f73e77448c43.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5b70b0a6-edb2-4743-bcfc-3022dc5ed931 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:53.847Z InvocationID=52641d6b-5f6e-4e83-a3e2-243a0a9ef788 ErrorCode=500 Starting to create Csar for component SecurityRules 2022-04-26T06:22:56.685Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=951e4b42-15c6-4ead-ab70-fc2172f73657 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:22:56.685Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=951e4b42-15c6-4ead-ab70-fc2172f73657 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:22:56.705Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "SecurityRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "4016522f-bbcc-4490-8913-6d4e342dec41" INVARIANT_UUID = "7712c948-72b7-4e7e-a04d-69de58b8005c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "dc1a6291-408a-4965-9f94-f73e77448c43.assettoscatemplate" CURR_ARTIFACT_UUID = "dc1a6291-408a-4965-9f94-f73e77448c43.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Securityrules-template.yml,0,dc1a6291-408a-4965-9f94-f73e77448c43.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5b70b0a6-edb2-4743-bcfc-3022dc5ed931 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:53.847Z InvocationID=52641d6b-5f6e-4e83-a3e2-243a0a9ef788 ErrorCode=500 Ended create Csar for component SecurityRules 2022-04-26T06:22:56.706Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "SecurityRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "4016522f-bbcc-4490-8913-6d4e342dec41" INVARIANT_UUID = "7712c948-72b7-4e7e-a04d-69de58b8005c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "dc1a6291-408a-4965-9f94-f73e77448c43.assettoscatemplate" CURR_ARTIFACT_UUID = "dc1a6291-408a-4965-9f94-f73e77448c43.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Securityrules-template.yml,0,dc1a6291-408a-4965-9f94-f73e77448c43.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5b70b0a6-edb2-4743-bcfc-3022dc5ed931 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:53.847Z InvocationID=52641d6b-5f6e-4e83-a3e2-243a0a9ef788 ErrorCode=500 lock resource with id dc1a6291-408a-4965-9f94-f73e77448c43 2022-04-26T06:22:57.460Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "SecurityRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "4016522f-bbcc-4490-8913-6d4e342dec41" INVARIANT_UUID = "7712c948-72b7-4e7e-a04d-69de58b8005c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "dc1a6291-408a-4965-9f94-f73e77448c43.assettoscacsar" CURR_ARTIFACT_UUID = "dc1a6291-408a-4965-9f94-f73e77448c43.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Securityrules-csar.csar,0,dc1a6291-408a-4965-9f94-f73e77448c43.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=3612 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ed5e63fc-8577-480d-a1c3-1e05546aa856 TargetServiceName=null EntryTimestamp=2022-04-26T06:22:53.847Z InvocationID=52641d6b-5f6e-4e83-a3e2-243a0a9ef788 ErrorCode=500 LogTimestamp=2022-04-26T06:22:57.459667Z ResponseDesc=Created Exiting. 2022-04-26T06:22:57.460Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:22:57.544Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=e53ef218-eb85-4b2a-9b86-25b99c1e6008 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:57.543696Z InvocationID=19f5e82a-54d6-4f67-83e8-193add7d3aaa LogTimestamp=2022-04-26T06:22:57.543789Z Entering 2022-04-26T06:22:57.544Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=e53ef218-eb85-4b2a-9b86-25b99c1e6008 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:22:57.543696Z InvocationID=19f5e82a-54d6-4f67-83e8-193add7d3aaa LogTimestamp=2022-04-26T06:22:57.543789Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:22:58.258Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:57.544Z InvocationID=41bdfa0e-49c6-4eed-b430-d1995f0ae08c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:58.258Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:57.544Z InvocationID=41bdfa0e-49c6-4eed-b430-d1995f0ae08c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:58.258Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:57.544Z InvocationID=41bdfa0e-49c6-4eed-b430-d1995f0ae08c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:58.258Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:57.544Z InvocationID=41bdfa0e-49c6-4eed-b430-d1995f0ae08c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:22:58.259Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:22:57.544Z InvocationID=41bdfa0e-49c6-4eed-b430-d1995f0ae08c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Contrailabstractsubstitute 2022-04-26T06:22:58.945Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96ffface-c1ee-4e42-80c1-10dd8067eed5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:22:58.946Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96ffface-c1ee-4e42-80c1-10dd8067eed5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f61d0b4 2022-04-26T06:22:58.946Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96ffface-c1ee-4e42-80c1-10dd8067eed5 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:22:58.946Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96ffface-c1ee-4e42-80c1-10dd8067eed5 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:22:59.152Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailAbstractSubstitute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2890de11-d8e6-45fb-95b8-042c5fd29e07" INVARIANT_UUID = "fa5ac0ee-1a9f-4c04-a508-c186c4c7ccd6" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.abstract.nodes.contrail.AbstractSubstitute" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2890de11-d8e6-45fb-95b8-042c5fd29e07 EntryTimestamp=2022-04-26T06:22:57.544Z InvocationID=41bdfa0e-49c6-4eed-b430-d1995f0ae08c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 22dba731-25e2-47e5-8235-1d6f2cec6351 2022-04-26T06:22:59.981Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=96ffface-c1ee-4e42-80c1-10dd8067eed5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:22:59.981Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=96ffface-c1ee-4e42-80c1-10dd8067eed5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:23:02.130Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=554d4ceb-3beb-4071-8e90-a8f0ed3d607d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:23:02.130Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=554d4ceb-3beb-4071-8e90-a8f0ed3d607d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5eb5017f 2022-04-26T06:23:02.134Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=554d4ceb-3beb-4071-8e90-a8f0ed3d607d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:23:02.134Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=554d4ceb-3beb-4071-8e90-a8f0ed3d607d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:23:02.152Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailAbstractSubstitute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2890de11-d8e6-45fb-95b8-042c5fd29e07" INVARIANT_UUID = "fa5ac0ee-1a9f-4c04-a508-c186c4c7ccd6" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "22dba731-25e2-47e5-8235-1d6f2cec6351.assettoscatemplate" CURR_ARTIFACT_UUID = "22dba731-25e2-47e5-8235-1d6f2cec6351.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailabstractsubstitute-template.yml,0,22dba731-25e2-47e5-8235-1d6f2cec6351.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=4607 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b9656f2a-ac10-4de2-9197-916a1e6410d4 EntryTimestamp=2022-04-26T06:22:57.544Z InvocationID=41bdfa0e-49c6-4eed-b430-d1995f0ae08c LogTimestamp=2022-04-26T06:23:02.151633Z ResponseDesc=Created Exiting. 2022-04-26T06:23:02.152Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:23:02.248Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f8d39011-77c7-436c-899b-c6b2523ee445 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:02.248372Z InvocationID=c38d362a-0a4a-4345-ab15-63b4a870fc6a LogTimestamp=2022-04-26T06:23:02.248488Z Entering 2022-04-26T06:23:02.249Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f8d39011-77c7-436c-899b-c6b2523ee445 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:02.248372Z InvocationID=c38d362a-0a4a-4345-ab15-63b4a870fc6a LogTimestamp=2022-04-26T06:23:02.248488Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:23:03.187Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=554d4ceb-3beb-4071-8e90-a8f0ed3d607d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:23:03.187Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=554d4ceb-3beb-4071-8e90-a8f0ed3d607d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:23:03.345Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:02.249Z InvocationID=552ec583-79f8-478d-ae6b-3c2ecb6a1bd0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:03.346Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:02.249Z InvocationID=552ec583-79f8-478d-ae6b-3c2ecb6a1bd0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:03.346Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:02.249Z InvocationID=552ec583-79f8-478d-ae6b-3c2ecb6a1bd0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:03.346Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:02.249Z InvocationID=552ec583-79f8-478d-ae6b-3c2ecb6a1bd0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:03.346Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:02.249Z InvocationID=552ec583-79f8-478d-ae6b-3c2ecb6a1bd0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Contrailcompute 2022-04-26T06:23:03.755Z [qtp2085002312-13] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailCompute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "766f976c-4097-4d97-b5a0-879ecf56fdab" INVARIANT_UUID = "adc6f9ef-8b55-4bf4-b549-63291dd9738b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.heat.contrail.Compute" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=766f976c-4097-4d97-b5a0-879ecf56fdab EntryTimestamp=2022-04-26T06:23:02.249Z InvocationID=552ec583-79f8-478d-ae6b-3c2ecb6a1bd0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:23:03.756Z [qtp2085002312-13] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailCompute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "766f976c-4097-4d97-b5a0-879ecf56fdab" INVARIANT_UUID = "adc6f9ef-8b55-4bf4-b549-63291dd9738b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.heat.contrail.Compute" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=766f976c-4097-4d97-b5a0-879ecf56fdab TargetServiceName=null EntryTimestamp=2022-04-26T06:23:02.249Z InvocationID=552ec583-79f8-478d-ae6b-3c2ecb6a1bd0 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:23:04.151Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailCompute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "766f976c-4097-4d97-b5a0-879ecf56fdab" INVARIANT_UUID = "adc6f9ef-8b55-4bf4-b549-63291dd9738b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.heat.contrail.Compute" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=766f976c-4097-4d97-b5a0-879ecf56fdab TargetServiceName=null EntryTimestamp=2022-04-26T06:23:02.249Z InvocationID=552ec583-79f8-478d-ae6b-3c2ecb6a1bd0 ErrorCode=500 lock resource with id 8194aa21-2498-4668-8a81-087d3667f208 2022-04-26T06:23:04.543Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ecf83bbe-1dac-4d73-9b5e-372a0686d22b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:23:04.543Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ecf83bbe-1dac-4d73-9b5e-372a0686d22b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7b345bb3 2022-04-26T06:23:04.544Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ecf83bbe-1dac-4d73-9b5e-372a0686d22b PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:23:04.544Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ecf83bbe-1dac-4d73-9b5e-372a0686d22b PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:23:05.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:23:05.378Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:23:05.546Z [qtp2085002312-13] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailCompute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "766f976c-4097-4d97-b5a0-879ecf56fdab" INVARIANT_UUID = "adc6f9ef-8b55-4bf4-b549-63291dd9738b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8194aa21-2498-4668-8a81-087d3667f208.assettoscatemplate" CURR_ARTIFACT_UUID = "8194aa21-2498-4668-8a81-087d3667f208.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailcompute-template.yml,0,8194aa21-2498-4668-8a81-087d3667f208.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1f398b9c-538f-4e00-8200-d6fa6e15238b TargetServiceName=null EntryTimestamp=2022-04-26T06:23:02.249Z InvocationID=552ec583-79f8-478d-ae6b-3c2ecb6a1bd0 ErrorCode=500 Starting to create Csar for component ContrailCompute 2022-04-26T06:23:05.581Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ecf83bbe-1dac-4d73-9b5e-372a0686d22b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:23:05.582Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ecf83bbe-1dac-4d73-9b5e-372a0686d22b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:23:05.704Z [qtp2085002312-13] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailCompute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "766f976c-4097-4d97-b5a0-879ecf56fdab" INVARIANT_UUID = "adc6f9ef-8b55-4bf4-b549-63291dd9738b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8194aa21-2498-4668-8a81-087d3667f208.assettoscatemplate" CURR_ARTIFACT_UUID = "8194aa21-2498-4668-8a81-087d3667f208.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailcompute-template.yml,0,8194aa21-2498-4668-8a81-087d3667f208.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1f398b9c-538f-4e00-8200-d6fa6e15238b TargetServiceName=null EntryTimestamp=2022-04-26T06:23:02.249Z InvocationID=552ec583-79f8-478d-ae6b-3c2ecb6a1bd0 ErrorCode=500 Ended create Csar for component ContrailCompute 2022-04-26T06:23:05.705Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailCompute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "766f976c-4097-4d97-b5a0-879ecf56fdab" INVARIANT_UUID = "adc6f9ef-8b55-4bf4-b549-63291dd9738b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8194aa21-2498-4668-8a81-087d3667f208.assettoscatemplate" CURR_ARTIFACT_UUID = "8194aa21-2498-4668-8a81-087d3667f208.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailcompute-template.yml,0,8194aa21-2498-4668-8a81-087d3667f208.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1f398b9c-538f-4e00-8200-d6fa6e15238b TargetServiceName=null EntryTimestamp=2022-04-26T06:23:02.249Z InvocationID=552ec583-79f8-478d-ae6b-3c2ecb6a1bd0 ErrorCode=500 lock resource with id 8194aa21-2498-4668-8a81-087d3667f208 2022-04-26T06:23:06.457Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailCompute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "766f976c-4097-4d97-b5a0-879ecf56fdab" INVARIANT_UUID = "adc6f9ef-8b55-4bf4-b549-63291dd9738b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8194aa21-2498-4668-8a81-087d3667f208.assettoscacsar" CURR_ARTIFACT_UUID = "8194aa21-2498-4668-8a81-087d3667f208.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Contrailcompute-csar.csar,0,8194aa21-2498-4668-8a81-087d3667f208.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=4208 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5035957f-a533-438a-a183-6b84fc89c600 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:02.249Z InvocationID=552ec583-79f8-478d-ae6b-3c2ecb6a1bd0 ErrorCode=500 LogTimestamp=2022-04-26T06:23:06.457514Z ResponseDesc=Created Exiting. 2022-04-26T06:23:06.459Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:23:06.475Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=7cad37f3-afb8-4094-b09a-d7c74b1ac4ef InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:06.475430Z InvocationID=c614c837-4c6a-4296-9220-7e92d088779f LogTimestamp=2022-04-26T06:23:06.475569Z Entering 2022-04-26T06:23:06.476Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=7cad37f3-afb8-4094-b09a-d7c74b1ac4ef InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:06.475430Z InvocationID=c614c837-4c6a-4296-9220-7e92d088779f LogTimestamp=2022-04-26T06:23:06.475569Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:23:07.546Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:06.476Z InvocationID=e4e18d2a-74f0-4820-be03-baeb87ea5021 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:07.546Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:06.476Z InvocationID=e4e18d2a-74f0-4820-be03-baeb87ea5021 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:07.547Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:06.476Z InvocationID=e4e18d2a-74f0-4820-be03-baeb87ea5021 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:07.547Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:06.476Z InvocationID=e4e18d2a-74f0-4820-be03-baeb87ea5021 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:07.630Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:06.476Z InvocationID=e4e18d2a-74f0-4820-be03-baeb87ea5021 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Contrailv2virtualmachineinterface 2022-04-26T06:23:07.647Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:23:07.842Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce6c10eb-7c7a-45a6-ba18-2cbcaccb2e82 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:23:07.842Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce6c10eb-7c7a-45a6-ba18-2cbcaccb2e82 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@65b4191d 2022-04-26T06:23:07.842Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce6c10eb-7c7a-45a6-ba18-2cbcaccb2e82 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:23:07.843Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce6c10eb-7c7a-45a6-ba18-2cbcaccb2e82 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:23:08.146Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailV2VirtualMachineInterface" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "df3977b3-373b-406d-8b53-1f99a1666c36" INVARIANT_UUID = "b53b6bcf-13b1-451e-848c-1ea43fc8ce40" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.contrailV2.VirtualMachineInterface" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=df3977b3-373b-406d-8b53-1f99a1666c36 EntryTimestamp=2022-04-26T06:23:06.476Z InvocationID=e4e18d2a-74f0-4820-be03-baeb87ea5021 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:23:08.147Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailV2VirtualMachineInterface" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "df3977b3-373b-406d-8b53-1f99a1666c36" INVARIANT_UUID = "b53b6bcf-13b1-451e-848c-1ea43fc8ce40" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.contrailV2.VirtualMachineInterface" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=df3977b3-373b-406d-8b53-1f99a1666c36 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:06.476Z InvocationID=e4e18d2a-74f0-4820-be03-baeb87ea5021 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:23:08.443Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailV2VirtualMachineInterface" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "df3977b3-373b-406d-8b53-1f99a1666c36" INVARIANT_UUID = "b53b6bcf-13b1-451e-848c-1ea43fc8ce40" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.contrailV2.VirtualMachineInterface" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=df3977b3-373b-406d-8b53-1f99a1666c36 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:06.476Z InvocationID=e4e18d2a-74f0-4820-be03-baeb87ea5021 ErrorCode=500 lock resource with id a078b6fb-2992-4afe-ab2c-58650f41238d 2022-04-26T06:23:08.873Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ce6c10eb-7c7a-45a6-ba18-2cbcaccb2e82 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:23:08.873Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ce6c10eb-7c7a-45a6-ba18-2cbcaccb2e82 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:23:09.848Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailV2VirtualMachineInterface" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "df3977b3-373b-406d-8b53-1f99a1666c36" INVARIANT_UUID = "b53b6bcf-13b1-451e-848c-1ea43fc8ce40" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a078b6fb-2992-4afe-ab2c-58650f41238d.assettoscatemplate" CURR_ARTIFACT_UUID = "a078b6fb-2992-4afe-ab2c-58650f41238d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2virtualmachineinterface-template.yml,0,a078b6fb-2992-4afe-ab2c-58650f41238d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f599c79e-eb8a-428e-92fb-77fb3da50cf1 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:06.476Z InvocationID=e4e18d2a-74f0-4820-be03-baeb87ea5021 ErrorCode=500 Starting to create Csar for component ContrailV2VirtualMachineInterface 2022-04-26T06:23:10.014Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailV2VirtualMachineInterface" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "df3977b3-373b-406d-8b53-1f99a1666c36" INVARIANT_UUID = "b53b6bcf-13b1-451e-848c-1ea43fc8ce40" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a078b6fb-2992-4afe-ab2c-58650f41238d.assettoscatemplate" CURR_ARTIFACT_UUID = "a078b6fb-2992-4afe-ab2c-58650f41238d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2virtualmachineinterface-template.yml,0,a078b6fb-2992-4afe-ab2c-58650f41238d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f599c79e-eb8a-428e-92fb-77fb3da50cf1 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:06.476Z InvocationID=e4e18d2a-74f0-4820-be03-baeb87ea5021 ErrorCode=500 Ended create Csar for component ContrailV2VirtualMachineInterface 2022-04-26T06:23:10.015Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailV2VirtualMachineInterface" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "df3977b3-373b-406d-8b53-1f99a1666c36" INVARIANT_UUID = "b53b6bcf-13b1-451e-848c-1ea43fc8ce40" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a078b6fb-2992-4afe-ab2c-58650f41238d.assettoscatemplate" CURR_ARTIFACT_UUID = "a078b6fb-2992-4afe-ab2c-58650f41238d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2virtualmachineinterface-template.yml,0,a078b6fb-2992-4afe-ab2c-58650f41238d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f599c79e-eb8a-428e-92fb-77fb3da50cf1 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:06.476Z InvocationID=e4e18d2a-74f0-4820-be03-baeb87ea5021 ErrorCode=500 lock resource with id a078b6fb-2992-4afe-ab2c-58650f41238d 2022-04-26T06:23:10.048Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4220ec32-7f8e-43c7-a3d4-a71eb745eadc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:23:10.049Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4220ec32-7f8e-43c7-a3d4-a71eb745eadc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5830556e 2022-04-26T06:23:10.049Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4220ec32-7f8e-43c7-a3d4-a71eb745eadc PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:23:10.049Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4220ec32-7f8e-43c7-a3d4-a71eb745eadc PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:23:10.658Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailV2VirtualMachineInterface" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "df3977b3-373b-406d-8b53-1f99a1666c36" INVARIANT_UUID = "b53b6bcf-13b1-451e-848c-1ea43fc8ce40" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a078b6fb-2992-4afe-ab2c-58650f41238d.assettoscacsar" CURR_ARTIFACT_UUID = "a078b6fb-2992-4afe-ab2c-58650f41238d.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Contrailv2virtualmachineinterface-csar.csar,0,a078b6fb-2992-4afe-ab2c-58650f41238d.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=4182 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1ac1a9e1-191d-45a7-9520-19636d20b346 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:06.476Z InvocationID=e4e18d2a-74f0-4820-be03-baeb87ea5021 ErrorCode=500 LogTimestamp=2022-04-26T06:23:10.658262Z ResponseDesc=Created Exiting. 2022-04-26T06:23:10.659Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:23:10.673Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=58623db8-5889-4461-86cb-8f2ab0c2d1b9 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:10.673027Z InvocationID=ffa83b6c-abde-49e4-8fb5-58cf4243bc55 LogTimestamp=2022-04-26T06:23:10.673140Z Entering 2022-04-26T06:23:10.673Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=58623db8-5889-4461-86cb-8f2ab0c2d1b9 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:10.673027Z InvocationID=ffa83b6c-abde-49e4-8fb5-58cf4243bc55 LogTimestamp=2022-04-26T06:23:10.673140Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:23:10.764Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:10.674Z InvocationID=9d00ef31-c444-4500-a432-14a51d36cd6e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:10.764Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:10.674Z InvocationID=9d00ef31-c444-4500-a432-14a51d36cd6e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:10.764Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:10.674Z InvocationID=9d00ef31-c444-4500-a432-14a51d36cd6e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:10.765Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:10.674Z InvocationID=9d00ef31-c444-4500-a432-14a51d36cd6e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:10.765Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:10.674Z InvocationID=9d00ef31-c444-4500-a432-14a51d36cd6e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Subinterface 2022-04-26T06:23:10.808Z [qtp2085002312-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "subInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "5bbc6d1a-1f24-4cf0-bb22-76638ef9b3cc" INVARIANT_UUID = "cbdb65c3-d964-4783-92c3-2be255ec431c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.network.SubInterface" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5bbc6d1a-1f24-4cf0-bb22-76638ef9b3cc EntryTimestamp=2022-04-26T06:23:10.674Z InvocationID=9d00ef31-c444-4500-a432-14a51d36cd6e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:23:10.809Z [qtp2085002312-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "subInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "5bbc6d1a-1f24-4cf0-bb22-76638ef9b3cc" INVARIANT_UUID = "cbdb65c3-d964-4783-92c3-2be255ec431c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.network.SubInterface" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5bbc6d1a-1f24-4cf0-bb22-76638ef9b3cc TargetServiceName=null EntryTimestamp=2022-04-26T06:23:10.674Z InvocationID=9d00ef31-c444-4500-a432-14a51d36cd6e ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:23:10.870Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "subInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "5bbc6d1a-1f24-4cf0-bb22-76638ef9b3cc" INVARIANT_UUID = "cbdb65c3-d964-4783-92c3-2be255ec431c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.network.SubInterface" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5bbc6d1a-1f24-4cf0-bb22-76638ef9b3cc TargetServiceName=null EntryTimestamp=2022-04-26T06:23:10.674Z InvocationID=9d00ef31-c444-4500-a432-14a51d36cd6e ErrorCode=500 lock resource with id 13d16adb-fcd9-428a-8861-c80608beb856 2022-04-26T06:23:11.089Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=4220ec32-7f8e-43c7-a3d4-a71eb745eadc ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:23:11.089Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=4220ec32-7f8e-43c7-a3d4-a71eb745eadc ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:23:11.481Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "subInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "5bbc6d1a-1f24-4cf0-bb22-76638ef9b3cc" INVARIANT_UUID = "cbdb65c3-d964-4783-92c3-2be255ec431c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "13d16adb-fcd9-428a-8861-c80608beb856.assettoscatemplate" CURR_ARTIFACT_UUID = "13d16adb-fcd9-428a-8861-c80608beb856.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Subinterface-template.yml,0,13d16adb-fcd9-428a-8861-c80608beb856.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1d1df7d8-d348-4595-9771-76f7fc6e93a4 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:10.674Z InvocationID=9d00ef31-c444-4500-a432-14a51d36cd6e ErrorCode=500 Starting to create Csar for component subInterface 2022-04-26T06:23:11.642Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "subInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "5bbc6d1a-1f24-4cf0-bb22-76638ef9b3cc" INVARIANT_UUID = "cbdb65c3-d964-4783-92c3-2be255ec431c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "13d16adb-fcd9-428a-8861-c80608beb856.assettoscatemplate" CURR_ARTIFACT_UUID = "13d16adb-fcd9-428a-8861-c80608beb856.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Subinterface-template.yml,0,13d16adb-fcd9-428a-8861-c80608beb856.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1d1df7d8-d348-4595-9771-76f7fc6e93a4 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:10.674Z InvocationID=9d00ef31-c444-4500-a432-14a51d36cd6e ErrorCode=500 Ended create Csar for component subInterface 2022-04-26T06:23:11.642Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "subInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "5bbc6d1a-1f24-4cf0-bb22-76638ef9b3cc" INVARIANT_UUID = "cbdb65c3-d964-4783-92c3-2be255ec431c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "13d16adb-fcd9-428a-8861-c80608beb856.assettoscatemplate" CURR_ARTIFACT_UUID = "13d16adb-fcd9-428a-8861-c80608beb856.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Subinterface-template.yml,0,13d16adb-fcd9-428a-8861-c80608beb856.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1d1df7d8-d348-4595-9771-76f7fc6e93a4 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:10.674Z InvocationID=9d00ef31-c444-4500-a432-14a51d36cd6e ErrorCode=500 lock resource with id 13d16adb-fcd9-428a-8861-c80608beb856 2022-04-26T06:23:12.199Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "subInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "5bbc6d1a-1f24-4cf0-bb22-76638ef9b3cc" INVARIANT_UUID = "cbdb65c3-d964-4783-92c3-2be255ec431c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "13d16adb-fcd9-428a-8861-c80608beb856.assettoscacsar" CURR_ARTIFACT_UUID = "13d16adb-fcd9-428a-8861-c80608beb856.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Subinterface-csar.csar,0,13d16adb-fcd9-428a-8861-c80608beb856.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1525 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bbaa789f-e31c-4c35-aa89-ccbd932dbd59 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:10.674Z InvocationID=9d00ef31-c444-4500-a432-14a51d36cd6e ErrorCode=500 LogTimestamp=2022-04-26T06:23:12.199001Z ResponseDesc=Created Exiting. 2022-04-26T06:23:12.199Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:23:12.221Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5beccfee-3843-40f3-8349-9d61ac098159 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:12.220804Z InvocationID=257b6b9d-bac6-4e76-9905-1a3a95ee230b LogTimestamp=2022-04-26T06:23:12.220889Z Entering 2022-04-26T06:23:12.221Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5beccfee-3843-40f3-8349-9d61ac098159 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:12.220804Z InvocationID=257b6b9d-bac6-4e76-9905-1a3a95ee230b LogTimestamp=2022-04-26T06:23:12.220889Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:23:12.385Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:12.221Z InvocationID=9ca7729a-1d6f-4c74-98aa-8bd71ab8ab2d PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:12.385Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:12.221Z InvocationID=9ca7729a-1d6f-4c74-98aa-8bd71ab8ab2d PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:12.385Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:12.221Z InvocationID=9ca7729a-1d6f-4c74-98aa-8bd71ab8ab2d PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:12.385Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:12.221Z InvocationID=9ca7729a-1d6f-4c74-98aa-8bd71ab8ab2d PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:12.392Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:12.221Z InvocationID=9ca7729a-1d6f-4c74-98aa-8bd71ab8ab2d PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Contrailv2vlansubinterface 2022-04-26T06:23:12.436Z [qtp2085002312-20] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "contrailV2VLANSubInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "45763403-ce92-45d0-b226-05a93669df5f" INVARIANT_UUID = "f7112d76-95e2-4309-8e09-3e4f5e1af970" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.network.contrailV2.VLANSubInterface" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=45763403-ce92-45d0-b226-05a93669df5f EntryTimestamp=2022-04-26T06:23:12.221Z InvocationID=9ca7729a-1d6f-4c74-98aa-8bd71ab8ab2d PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:23:12.437Z [qtp2085002312-20] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "contrailV2VLANSubInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "45763403-ce92-45d0-b226-05a93669df5f" INVARIANT_UUID = "f7112d76-95e2-4309-8e09-3e4f5e1af970" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.network.contrailV2.VLANSubInterface" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=45763403-ce92-45d0-b226-05a93669df5f TargetServiceName=null EntryTimestamp=2022-04-26T06:23:12.221Z InvocationID=9ca7729a-1d6f-4c74-98aa-8bd71ab8ab2d ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:23:12.479Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "contrailV2VLANSubInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "45763403-ce92-45d0-b226-05a93669df5f" INVARIANT_UUID = "f7112d76-95e2-4309-8e09-3e4f5e1af970" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.network.contrailV2.VLANSubInterface" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=45763403-ce92-45d0-b226-05a93669df5f TargetServiceName=null EntryTimestamp=2022-04-26T06:23:12.221Z InvocationID=9ca7729a-1d6f-4c74-98aa-8bd71ab8ab2d ErrorCode=500 lock resource with id 9005983f-987b-4bb0-bfca-269e5c24668f 2022-04-26T06:23:12.786Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b185eee7-25ce-4106-b226-18b796c008c3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:23:12.786Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b185eee7-25ce-4106-b226-18b796c008c3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6ce3a127 2022-04-26T06:23:12.786Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b185eee7-25ce-4106-b226-18b796c008c3 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:23:12.786Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b185eee7-25ce-4106-b226-18b796c008c3 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:23:13.141Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "contrailV2VLANSubInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "45763403-ce92-45d0-b226-05a93669df5f" INVARIANT_UUID = "f7112d76-95e2-4309-8e09-3e4f5e1af970" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "9005983f-987b-4bb0-bfca-269e5c24668f.assettoscatemplate" CURR_ARTIFACT_UUID = "9005983f-987b-4bb0-bfca-269e5c24668f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2vlansubinterface-template.yml,0,9005983f-987b-4bb0-bfca-269e5c24668f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3adffa15-9b79-4b62-abee-6a53667c355f TargetServiceName=null EntryTimestamp=2022-04-26T06:23:12.221Z InvocationID=9ca7729a-1d6f-4c74-98aa-8bd71ab8ab2d ErrorCode=500 Starting to create Csar for component contrailV2VLANSubInterface 2022-04-26T06:23:13.351Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "contrailV2VLANSubInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "45763403-ce92-45d0-b226-05a93669df5f" INVARIANT_UUID = "f7112d76-95e2-4309-8e09-3e4f5e1af970" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "9005983f-987b-4bb0-bfca-269e5c24668f.assettoscatemplate" CURR_ARTIFACT_UUID = "9005983f-987b-4bb0-bfca-269e5c24668f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2vlansubinterface-template.yml,0,9005983f-987b-4bb0-bfca-269e5c24668f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3adffa15-9b79-4b62-abee-6a53667c355f TargetServiceName=null EntryTimestamp=2022-04-26T06:23:12.221Z InvocationID=9ca7729a-1d6f-4c74-98aa-8bd71ab8ab2d ErrorCode=500 Ended create Csar for component contrailV2VLANSubInterface 2022-04-26T06:23:13.351Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "contrailV2VLANSubInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "45763403-ce92-45d0-b226-05a93669df5f" INVARIANT_UUID = "f7112d76-95e2-4309-8e09-3e4f5e1af970" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "9005983f-987b-4bb0-bfca-269e5c24668f.assettoscatemplate" CURR_ARTIFACT_UUID = "9005983f-987b-4bb0-bfca-269e5c24668f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2vlansubinterface-template.yml,0,9005983f-987b-4bb0-bfca-269e5c24668f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3adffa15-9b79-4b62-abee-6a53667c355f TargetServiceName=null EntryTimestamp=2022-04-26T06:23:12.221Z InvocationID=9ca7729a-1d6f-4c74-98aa-8bd71ab8ab2d ErrorCode=500 lock resource with id 9005983f-987b-4bb0-bfca-269e5c24668f 2022-04-26T06:23:13.808Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=b185eee7-25ce-4106-b226-18b796c008c3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:23:13.809Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=b185eee7-25ce-4106-b226-18b796c008c3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:23:13.925Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "contrailV2VLANSubInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "45763403-ce92-45d0-b226-05a93669df5f" INVARIANT_UUID = "f7112d76-95e2-4309-8e09-3e4f5e1af970" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "9005983f-987b-4bb0-bfca-269e5c24668f.assettoscacsar" CURR_ARTIFACT_UUID = "9005983f-987b-4bb0-bfca-269e5c24668f.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Contrailv2vlansubinterface-csar.csar,0,9005983f-987b-4bb0-bfca-269e5c24668f.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1704 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a3ed1bbf-a103-4702-b54c-e459cf521a61 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:12.221Z InvocationID=9ca7729a-1d6f-4c74-98aa-8bd71ab8ab2d ErrorCode=500 LogTimestamp=2022-04-26T06:23:13.925021Z ResponseDesc=Created Exiting. 2022-04-26T06:23:13.925Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:23:13.940Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c04967bc-2a58-4bc0-bccb-9241f05deb80 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:13.940406Z InvocationID=2d018b05-b16f-4977-aacc-c61156ab7478 LogTimestamp=2022-04-26T06:23:13.940474Z Entering 2022-04-26T06:23:13.940Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c04967bc-2a58-4bc0-bccb-9241f05deb80 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:13.940406Z InvocationID=2d018b05-b16f-4977-aacc-c61156ab7478 LogTimestamp=2022-04-26T06:23:13.940474Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:23:14.056Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:13.941Z InvocationID=49a104b0-de1e-4591-ab2e-8b9a1ac9a129 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:14.057Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:13.941Z InvocationID=49a104b0-de1e-4591-ab2e-8b9a1ac9a129 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:14.057Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:13.941Z InvocationID=49a104b0-de1e-4591-ab2e-8b9a1ac9a129 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:14.057Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:13.941Z InvocationID=49a104b0-de1e-4591-ab2e-8b9a1ac9a129 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:14.057Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:13.941Z InvocationID=49a104b0-de1e-4591-ab2e-8b9a1ac9a129 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Multiflavorvfc 2022-04-26T06:23:14.186Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "multiFlavorVFC" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "7d8961b1-61f1-458a-aea6-ae5a3c2c406e" INVARIANT_UUID = "f5c97ed6-1c38-4d16-a475-24f09c9a6eb9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.abstract.nodes.MultiFlavorVFC" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7d8961b1-61f1-458a-aea6-ae5a3c2c406e EntryTimestamp=2022-04-26T06:23:13.941Z InvocationID=49a104b0-de1e-4591-ab2e-8b9a1ac9a129 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 44fb551f-01a3-4265-b1e6-df605d601b5c 2022-04-26T06:23:14.852Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "multiFlavorVFC" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "7d8961b1-61f1-458a-aea6-ae5a3c2c406e" INVARIANT_UUID = "f5c97ed6-1c38-4d16-a475-24f09c9a6eb9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "44fb551f-01a3-4265-b1e6-df605d601b5c.assettoscatemplate" CURR_ARTIFACT_UUID = "44fb551f-01a3-4265-b1e6-df605d601b5c.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Multiflavorvfc-template.yml,0,44fb551f-01a3-4265-b1e6-df605d601b5c.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=911 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b8132cd6-f04d-46f2-87bc-06a7b47e06d2 EntryTimestamp=2022-04-26T06:23:13.941Z InvocationID=49a104b0-de1e-4591-ab2e-8b9a1ac9a129 LogTimestamp=2022-04-26T06:23:14.852416Z ResponseDesc=Created Exiting. 2022-04-26T06:23:14.853Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:23:14.864Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=30d5bd64-bd7c-4a4d-a9f7-3e7d55220775 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:14.864256Z InvocationID=4930155c-a133-4798-8f40-df88adae560d LogTimestamp=2022-04-26T06:23:14.864347Z Entering 2022-04-26T06:23:14.865Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=30d5bd64-bd7c-4a4d-a9f7-3e7d55220775 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:14.864256Z InvocationID=4930155c-a133-4798-8f40-df88adae560d LogTimestamp=2022-04-26T06:23:14.864347Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:23:14.968Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:14.865Z InvocationID=6a7ff25d-1b37-4224-9a97-55949345106a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:14.968Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:14.865Z InvocationID=6a7ff25d-1b37-4224-9a97-55949345106a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:14.968Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:14.865Z InvocationID=6a7ff25d-1b37-4224-9a97-55949345106a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:14.968Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:14.865Z InvocationID=6a7ff25d-1b37-4224-9a97-55949345106a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:14.968Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:14.865Z InvocationID=6a7ff25d-1b37-4224-9a97-55949345106a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Vnfconfiguration 2022-04-26T06:23:15.049Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "vnfConfiguration" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9aa02d36-98a6-4d7d-a1bb-ecdd2a8c70e0" INVARIANT_UUID = "a509c8df-5373-449c-8831-6047abb6e8c0" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.abstract.nodes.VnfConfiguration" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9aa02d36-98a6-4d7d-a1bb-ecdd2a8c70e0 EntryTimestamp=2022-04-26T06:23:14.865Z InvocationID=6a7ff25d-1b37-4224-9a97-55949345106a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 1d419b2f-109e-447f-a1ab-e9a6b968a9b1 2022-04-26T06:23:15.657Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "vnfConfiguration" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9aa02d36-98a6-4d7d-a1bb-ecdd2a8c70e0" INVARIANT_UUID = "a509c8df-5373-449c-8831-6047abb6e8c0" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1d419b2f-109e-447f-a1ab-e9a6b968a9b1.assettoscatemplate" CURR_ARTIFACT_UUID = "1d419b2f-109e-447f-a1ab-e9a6b968a9b1.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnfconfiguration-template.yml,0,1d419b2f-109e-447f-a1ab-e9a6b968a9b1.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=792 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=510a4851-c4b7-4bee-afbb-183b103b102b EntryTimestamp=2022-04-26T06:23:14.865Z InvocationID=6a7ff25d-1b37-4224-9a97-55949345106a LogTimestamp=2022-04-26T06:23:15.657416Z ResponseDesc=Created Exiting. 2022-04-26T06:23:15.658Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:23:15.668Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=cbd620e7-930e-4dbd-949f-1a37ea60bbd9 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:15.668417Z InvocationID=0d31dcd5-2842-41cc-ac03-92ea1a7e5f80 LogTimestamp=2022-04-26T06:23:15.668506Z Entering 2022-04-26T06:23:15.669Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=cbd620e7-930e-4dbd-949f-1a37ea60bbd9 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:15.668417Z InvocationID=0d31dcd5-2842-41cc-ac03-92ea1a7e5f80 LogTimestamp=2022-04-26T06:23:15.668506Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:23:15.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56058d2f-d720-4276-ba02-3ef1b876513c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:23:15.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56058d2f-d720-4276-ba02-3ef1b876513c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3459e992 2022-04-26T06:23:15.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56058d2f-d720-4276-ba02-3ef1b876513c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:23:15.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56058d2f-d720-4276-ba02-3ef1b876513c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:23:15.861Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:15.669Z InvocationID=1310d10b-7c39-4a2c-8920-835daa00239a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:15.861Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:15.669Z InvocationID=1310d10b-7c39-4a2c-8920-835daa00239a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:15.861Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:15.669Z InvocationID=1310d10b-7c39-4a2c-8920-835daa00239a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:15.861Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:15.669Z InvocationID=1310d10b-7c39-4a2c-8920-835daa00239a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:15.868Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:15.669Z InvocationID=1310d10b-7c39-4a2c-8920-835daa00239a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Extcp2 2022-04-26T06:23:15.944Z [qtp2085002312-13] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ExtCP2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "168a303b-b957-411e-9cf4-6926a070add6" INVARIANT_UUID = "fd9f8463-4068-47ce-a5a7-1ceb18064a2e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.v2.extCP" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=168a303b-b957-411e-9cf4-6926a070add6 EntryTimestamp=2022-04-26T06:23:15.669Z InvocationID=1310d10b-7c39-4a2c-8920-835daa00239a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:23:15.945Z [qtp2085002312-13] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ExtCP2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "168a303b-b957-411e-9cf4-6926a070add6" INVARIANT_UUID = "fd9f8463-4068-47ce-a5a7-1ceb18064a2e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.v2.extCP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=168a303b-b957-411e-9cf4-6926a070add6 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:15.669Z InvocationID=1310d10b-7c39-4a2c-8920-835daa00239a ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:23:15.976Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ExtCP2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "168a303b-b957-411e-9cf4-6926a070add6" INVARIANT_UUID = "fd9f8463-4068-47ce-a5a7-1ceb18064a2e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.v2.extCP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=168a303b-b957-411e-9cf4-6926a070add6 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:15.669Z InvocationID=1310d10b-7c39-4a2c-8920-835daa00239a ErrorCode=500 lock resource with id fbcb2e88-8e8b-4607-acb4-bca474f62ffd 2022-04-26T06:23:16.567Z [qtp2085002312-13] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ExtCP2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "168a303b-b957-411e-9cf4-6926a070add6" INVARIANT_UUID = "fd9f8463-4068-47ce-a5a7-1ceb18064a2e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "fbcb2e88-8e8b-4607-acb4-bca474f62ffd.assettoscatemplate" CURR_ARTIFACT_UUID = "fbcb2e88-8e8b-4607-acb4-bca474f62ffd.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extcp2-template.yml,0,fbcb2e88-8e8b-4607-acb4-bca474f62ffd.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a5bb4926-4c9f-4d49-b2ca-17329b9b4d05 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:15.669Z InvocationID=1310d10b-7c39-4a2c-8920-835daa00239a ErrorCode=500 Starting to create Csar for component ExtCP2 2022-04-26T06:23:16.719Z [qtp2085002312-13] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ExtCP2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "168a303b-b957-411e-9cf4-6926a070add6" INVARIANT_UUID = "fd9f8463-4068-47ce-a5a7-1ceb18064a2e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "fbcb2e88-8e8b-4607-acb4-bca474f62ffd.assettoscatemplate" CURR_ARTIFACT_UUID = "fbcb2e88-8e8b-4607-acb4-bca474f62ffd.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extcp2-template.yml,0,fbcb2e88-8e8b-4607-acb4-bca474f62ffd.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a5bb4926-4c9f-4d49-b2ca-17329b9b4d05 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:15.669Z InvocationID=1310d10b-7c39-4a2c-8920-835daa00239a ErrorCode=500 Ended create Csar for component ExtCP2 2022-04-26T06:23:16.720Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ExtCP2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "168a303b-b957-411e-9cf4-6926a070add6" INVARIANT_UUID = "fd9f8463-4068-47ce-a5a7-1ceb18064a2e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "fbcb2e88-8e8b-4607-acb4-bca474f62ffd.assettoscatemplate" CURR_ARTIFACT_UUID = "fbcb2e88-8e8b-4607-acb4-bca474f62ffd.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extcp2-template.yml,0,fbcb2e88-8e8b-4607-acb4-bca474f62ffd.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a5bb4926-4c9f-4d49-b2ca-17329b9b4d05 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:15.669Z InvocationID=1310d10b-7c39-4a2c-8920-835daa00239a ErrorCode=500 lock resource with id fbcb2e88-8e8b-4607-acb4-bca474f62ffd 2022-04-26T06:23:16.881Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=56058d2f-d720-4276-ba02-3ef1b876513c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:23:16.882Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=56058d2f-d720-4276-ba02-3ef1b876513c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:23:17.283Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ExtCP2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "168a303b-b957-411e-9cf4-6926a070add6" INVARIANT_UUID = "fd9f8463-4068-47ce-a5a7-1ceb18064a2e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "fbcb2e88-8e8b-4607-acb4-bca474f62ffd.assettoscacsar" CURR_ARTIFACT_UUID = "fbcb2e88-8e8b-4607-acb4-bca474f62ffd.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Extcp2-csar.csar,0,fbcb2e88-8e8b-4607-acb4-bca474f62ffd.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1613 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c653dd12-8ab3-4628-a9bd-403d01a8a0b6 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:15.669Z InvocationID=1310d10b-7c39-4a2c-8920-835daa00239a ErrorCode=500 LogTimestamp=2022-04-26T06:23:17.282955Z ResponseDesc=Created Exiting. 2022-04-26T06:23:17.284Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:23:17.296Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c6898ad4-8eb1-4006-a038-89719d5d1828 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:17.296455Z InvocationID=70e98c94-4d30-4a3e-a0fd-9659ebd5736b LogTimestamp=2022-04-26T06:23:17.296532Z Entering 2022-04-26T06:23:17.297Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c6898ad4-8eb1-4006-a038-89719d5d1828 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:17.296455Z InvocationID=70e98c94-4d30-4a3e-a0fd-9659ebd5736b LogTimestamp=2022-04-26T06:23:17.296532Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:23:17.425Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:17.297Z InvocationID=ccba7623-2d07-4b61-98d2-fcdb1ef37b60 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:17.426Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:17.297Z InvocationID=ccba7623-2d07-4b61-98d2-fcdb1ef37b60 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:17.426Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:17.297Z InvocationID=ccba7623-2d07-4b61-98d2-fcdb1ef37b60 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:17.426Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:17.297Z InvocationID=ccba7623-2d07-4b61-98d2-fcdb1ef37b60 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:17.433Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:17.297Z InvocationID=ccba7623-2d07-4b61-98d2-fcdb1ef37b60 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Extneutroncp 2022-04-26T06:23:17.477Z [qtp2085002312-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "extNeutronCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3066e3c5-3948-4d08-b5b9-34ee30326d8e" INVARIANT_UUID = "f355df07-05a0-40dc-961a-54f58aabb18b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.v2.extNeutronCP" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3066e3c5-3948-4d08-b5b9-34ee30326d8e EntryTimestamp=2022-04-26T06:23:17.297Z InvocationID=ccba7623-2d07-4b61-98d2-fcdb1ef37b60 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:23:17.478Z [qtp2085002312-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "extNeutronCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3066e3c5-3948-4d08-b5b9-34ee30326d8e" INVARIANT_UUID = "f355df07-05a0-40dc-961a-54f58aabb18b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.v2.extNeutronCP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3066e3c5-3948-4d08-b5b9-34ee30326d8e TargetServiceName=null EntryTimestamp=2022-04-26T06:23:17.297Z InvocationID=ccba7623-2d07-4b61-98d2-fcdb1ef37b60 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:23:17.513Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "extNeutronCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3066e3c5-3948-4d08-b5b9-34ee30326d8e" INVARIANT_UUID = "f355df07-05a0-40dc-961a-54f58aabb18b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.v2.extNeutronCP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3066e3c5-3948-4d08-b5b9-34ee30326d8e TargetServiceName=null EntryTimestamp=2022-04-26T06:23:17.297Z InvocationID=ccba7623-2d07-4b61-98d2-fcdb1ef37b60 ErrorCode=500 lock resource with id 688fdfbc-e49d-4454-952d-13ed3b87b965 2022-04-26T06:23:18.158Z [qtp2085002312-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "extNeutronCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3066e3c5-3948-4d08-b5b9-34ee30326d8e" INVARIANT_UUID = "f355df07-05a0-40dc-961a-54f58aabb18b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "688fdfbc-e49d-4454-952d-13ed3b87b965.assettoscatemplate" CURR_ARTIFACT_UUID = "688fdfbc-e49d-4454-952d-13ed3b87b965.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extneutroncp-template.yml,0,688fdfbc-e49d-4454-952d-13ed3b87b965.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c2afdcc8-6e34-4106-b9b4-2b1862c889e4 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:17.297Z InvocationID=ccba7623-2d07-4b61-98d2-fcdb1ef37b60 ErrorCode=500 Starting to create Csar for component extNeutronCP 2022-04-26T06:23:18.311Z [qtp2085002312-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "extNeutronCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3066e3c5-3948-4d08-b5b9-34ee30326d8e" INVARIANT_UUID = "f355df07-05a0-40dc-961a-54f58aabb18b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "688fdfbc-e49d-4454-952d-13ed3b87b965.assettoscatemplate" CURR_ARTIFACT_UUID = "688fdfbc-e49d-4454-952d-13ed3b87b965.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extneutroncp-template.yml,0,688fdfbc-e49d-4454-952d-13ed3b87b965.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c2afdcc8-6e34-4106-b9b4-2b1862c889e4 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:17.297Z InvocationID=ccba7623-2d07-4b61-98d2-fcdb1ef37b60 ErrorCode=500 Ended create Csar for component extNeutronCP 2022-04-26T06:23:18.312Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "extNeutronCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3066e3c5-3948-4d08-b5b9-34ee30326d8e" INVARIANT_UUID = "f355df07-05a0-40dc-961a-54f58aabb18b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "688fdfbc-e49d-4454-952d-13ed3b87b965.assettoscatemplate" CURR_ARTIFACT_UUID = "688fdfbc-e49d-4454-952d-13ed3b87b965.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extneutroncp-template.yml,0,688fdfbc-e49d-4454-952d-13ed3b87b965.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c2afdcc8-6e34-4106-b9b4-2b1862c889e4 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:17.297Z InvocationID=ccba7623-2d07-4b61-98d2-fcdb1ef37b60 ErrorCode=500 lock resource with id 688fdfbc-e49d-4454-952d-13ed3b87b965 2022-04-26T06:23:18.801Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24562113-ad56-40d1-afa6-9a5cb773a1ff PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:23:18.801Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24562113-ad56-40d1-afa6-9a5cb773a1ff PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3d2f3968 2022-04-26T06:23:18.801Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24562113-ad56-40d1-afa6-9a5cb773a1ff PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:23:18.801Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24562113-ad56-40d1-afa6-9a5cb773a1ff PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:23:18.873Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "extNeutronCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3066e3c5-3948-4d08-b5b9-34ee30326d8e" INVARIANT_UUID = "f355df07-05a0-40dc-961a-54f58aabb18b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "688fdfbc-e49d-4454-952d-13ed3b87b965.assettoscacsar" CURR_ARTIFACT_UUID = "688fdfbc-e49d-4454-952d-13ed3b87b965.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Extneutroncp-csar.csar,0,688fdfbc-e49d-4454-952d-13ed3b87b965.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1576 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=fbe22242-ea60-4f4c-91ba-a61119d2b381 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:17.297Z InvocationID=ccba7623-2d07-4b61-98d2-fcdb1ef37b60 ErrorCode=500 LogTimestamp=2022-04-26T06:23:18.873459Z ResponseDesc=Created Exiting. 2022-04-26T06:23:18.874Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:23:18.887Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5fde719a-b184-4aec-847b-2dc0fc27c1d1 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:18.887143Z InvocationID=6dc4c518-b67b-480d-850c-3ce0f706703f LogTimestamp=2022-04-26T06:23:18.887233Z Entering 2022-04-26T06:23:18.887Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5fde719a-b184-4aec-847b-2dc0fc27c1d1 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:18.887143Z InvocationID=6dc4c518-b67b-480d-850c-3ce0f706703f LogTimestamp=2022-04-26T06:23:18.887233Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:23:18.973Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:18.888Z InvocationID=f800cf94-6985-4c19-9dce-16b70df5f9bd PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:18.974Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:18.888Z InvocationID=f800cf94-6985-4c19-9dce-16b70df5f9bd PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:18.974Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:18.888Z InvocationID=f800cf94-6985-4c19-9dce-16b70df5f9bd PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:18.974Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:18.888Z InvocationID=f800cf94-6985-4c19-9dce-16b70df5f9bd PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:18.974Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:18.888Z InvocationID=f800cf94-6985-4c19-9dce-16b70df5f9bd PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Extcontrailcp 2022-04-26T06:23:19.011Z [qtp2085002312-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "extContrailCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "b776f9c2-6b70-416e-adf9-1121d472d17d" INVARIANT_UUID = "ef4536bb-e57b-4567-9fb5-cea03cc1941d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.v2.extContrailCP" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b776f9c2-6b70-416e-adf9-1121d472d17d EntryTimestamp=2022-04-26T06:23:18.888Z InvocationID=f800cf94-6985-4c19-9dce-16b70df5f9bd PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:23:19.012Z [qtp2085002312-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "extContrailCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "b776f9c2-6b70-416e-adf9-1121d472d17d" INVARIANT_UUID = "ef4536bb-e57b-4567-9fb5-cea03cc1941d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.v2.extContrailCP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b776f9c2-6b70-416e-adf9-1121d472d17d TargetServiceName=null EntryTimestamp=2022-04-26T06:23:18.888Z InvocationID=f800cf94-6985-4c19-9dce-16b70df5f9bd ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:23:19.043Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "extContrailCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "b776f9c2-6b70-416e-adf9-1121d472d17d" INVARIANT_UUID = "ef4536bb-e57b-4567-9fb5-cea03cc1941d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.v2.extContrailCP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b776f9c2-6b70-416e-adf9-1121d472d17d TargetServiceName=null EntryTimestamp=2022-04-26T06:23:18.888Z InvocationID=f800cf94-6985-4c19-9dce-16b70df5f9bd ErrorCode=500 lock resource with id df976228-02e1-448e-bd39-7bd2f7d8c9a8 2022-04-26T06:23:19.824Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=24562113-ad56-40d1-afa6-9a5cb773a1ff ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:23:19.825Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=24562113-ad56-40d1-afa6-9a5cb773a1ff ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:23:19.947Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "extContrailCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "b776f9c2-6b70-416e-adf9-1121d472d17d" INVARIANT_UUID = "ef4536bb-e57b-4567-9fb5-cea03cc1941d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "df976228-02e1-448e-bd39-7bd2f7d8c9a8.assettoscatemplate" CURR_ARTIFACT_UUID = "df976228-02e1-448e-bd39-7bd2f7d8c9a8.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extcontrailcp-template.yml,0,df976228-02e1-448e-bd39-7bd2f7d8c9a8.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=85982547-ac83-4730-9e3c-fc242ca9451f TargetServiceName=null EntryTimestamp=2022-04-26T06:23:18.888Z InvocationID=f800cf94-6985-4c19-9dce-16b70df5f9bd ErrorCode=500 Starting to create Csar for component extContrailCP 2022-04-26T06:23:20.101Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "extContrailCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "b776f9c2-6b70-416e-adf9-1121d472d17d" INVARIANT_UUID = "ef4536bb-e57b-4567-9fb5-cea03cc1941d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "df976228-02e1-448e-bd39-7bd2f7d8c9a8.assettoscatemplate" CURR_ARTIFACT_UUID = "df976228-02e1-448e-bd39-7bd2f7d8c9a8.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extcontrailcp-template.yml,0,df976228-02e1-448e-bd39-7bd2f7d8c9a8.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=85982547-ac83-4730-9e3c-fc242ca9451f TargetServiceName=null EntryTimestamp=2022-04-26T06:23:18.888Z InvocationID=f800cf94-6985-4c19-9dce-16b70df5f9bd ErrorCode=500 Ended create Csar for component extContrailCP 2022-04-26T06:23:20.102Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "extContrailCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "b776f9c2-6b70-416e-adf9-1121d472d17d" INVARIANT_UUID = "ef4536bb-e57b-4567-9fb5-cea03cc1941d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "df976228-02e1-448e-bd39-7bd2f7d8c9a8.assettoscatemplate" CURR_ARTIFACT_UUID = "df976228-02e1-448e-bd39-7bd2f7d8c9a8.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extcontrailcp-template.yml,0,df976228-02e1-448e-bd39-7bd2f7d8c9a8.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=85982547-ac83-4730-9e3c-fc242ca9451f TargetServiceName=null EntryTimestamp=2022-04-26T06:23:18.888Z InvocationID=f800cf94-6985-4c19-9dce-16b70df5f9bd ErrorCode=500 lock resource with id df976228-02e1-448e-bd39-7bd2f7d8c9a8 2022-04-26T06:23:20.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:23:20.381Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:23:21.156Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "extContrailCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "b776f9c2-6b70-416e-adf9-1121d472d17d" INVARIANT_UUID = "ef4536bb-e57b-4567-9fb5-cea03cc1941d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "df976228-02e1-448e-bd39-7bd2f7d8c9a8.assettoscacsar" CURR_ARTIFACT_UUID = "df976228-02e1-448e-bd39-7bd2f7d8c9a8.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Extcontrailcp-csar.csar,0,df976228-02e1-448e-bd39-7bd2f7d8c9a8.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=2267 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=08480f3e-0518-4407-9ff5-596f609a9cf2 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:18.888Z InvocationID=f800cf94-6985-4c19-9dce-16b70df5f9bd ErrorCode=500 LogTimestamp=2022-04-26T06:23:21.155977Z ResponseDesc=Created Exiting. 2022-04-26T06:23:21.156Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:23:21.243Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=bd89806b-0e51-4edc-bab7-cee20645df53 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:21.243535Z InvocationID=8737f8c5-357c-4ff3-8ccf-4674f517f485 LogTimestamp=2022-04-26T06:23:21.243613Z Entering 2022-04-26T06:23:21.244Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=bd89806b-0e51-4edc-bab7-cee20645df53 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:21.243535Z InvocationID=8737f8c5-357c-4ff3-8ccf-4674f517f485 LogTimestamp=2022-04-26T06:23:21.243613Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:23:21.944Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:21.244Z InvocationID=19f98b18-b2f9-42b2-b754-2538b377727f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:21.944Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:21.244Z InvocationID=19f98b18-b2f9-42b2-b754-2538b377727f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:21.944Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:21.244Z InvocationID=19f98b18-b2f9-42b2-b754-2538b377727f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:21.944Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:21.244Z InvocationID=19f98b18-b2f9-42b2-b754-2538b377727f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:21.949Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:21.244Z InvocationID=19f98b18-b2f9-42b2-b754-2538b377727f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name PortMirroringConfigurationByPolicy 2022-04-26T06:23:22.445Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b8ff63e-086e-4204-b030-f7bcfbb44716 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:23:22.445Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b8ff63e-086e-4204-b030-f7bcfbb44716 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6a25029f 2022-04-26T06:23:22.445Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b8ff63e-086e-4204-b030-f7bcfbb44716 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:23:22.445Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b8ff63e-086e-4204-b030-f7bcfbb44716 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:23:22.550Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Port Mirroring Configuration By Policy" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "2e39e2c1-9834-4ef8-acfa-77116ce76418" INVARIANT_UUID = "0b6c81f5-0b38-4bec-bf40-f7a494ca1ca3" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.nodes.PortMirroringConfigurationByPolicy" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2e39e2c1-9834-4ef8-acfa-77116ce76418 EntryTimestamp=2022-04-26T06:23:21.244Z InvocationID=19f98b18-b2f9-42b2-b754-2538b377727f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id ddb99ebb-12d2-4e92-a63b-4b442bed056e 2022-04-26T06:23:23.469Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=0b8ff63e-086e-4204-b030-f7bcfbb44716 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:23:23.469Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=0b8ff63e-086e-4204-b030-f7bcfbb44716 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:23:23.654Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Port Mirroring Configuration By Policy" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "2e39e2c1-9834-4ef8-acfa-77116ce76418" INVARIANT_UUID = "0b6c81f5-0b38-4bec-bf40-f7a494ca1ca3" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ddb99ebb-12d2-4e92-a63b-4b442bed056e.assettoscatemplate" CURR_ARTIFACT_UUID = "ddb99ebb-12d2-4e92-a63b-4b442bed056e.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-PortMirroringConfigurationByPolicy-template.yml,0,ddb99ebb-12d2-4e92-a63b-4b442bed056e.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=504924bc-fad2-4165-9729-61dc20768766 EntryTimestamp=2022-04-26T06:23:21.244Z InvocationID=19f98b18-b2f9-42b2-b754-2538b377727f PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Starting to create Csar for component Port Mirroring Configuration By Policy 2022-04-26T06:23:23.852Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Port Mirroring Configuration By Policy" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "2e39e2c1-9834-4ef8-acfa-77116ce76418" INVARIANT_UUID = "0b6c81f5-0b38-4bec-bf40-f7a494ca1ca3" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ddb99ebb-12d2-4e92-a63b-4b442bed056e.assettoscatemplate" CURR_ARTIFACT_UUID = "ddb99ebb-12d2-4e92-a63b-4b442bed056e.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-PortMirroringConfigurationByPolicy-template.yml,0,ddb99ebb-12d2-4e92-a63b-4b442bed056e.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=504924bc-fad2-4165-9729-61dc20768766 EntryTimestamp=2022-04-26T06:23:21.244Z InvocationID=19f98b18-b2f9-42b2-b754-2538b377727f Ended create Csar for component Port Mirroring Configuration By Policy 2022-04-26T06:23:23.853Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Port Mirroring Configuration By Policy" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "2e39e2c1-9834-4ef8-acfa-77116ce76418" INVARIANT_UUID = "0b6c81f5-0b38-4bec-bf40-f7a494ca1ca3" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ddb99ebb-12d2-4e92-a63b-4b442bed056e.assettoscatemplate" CURR_ARTIFACT_UUID = "ddb99ebb-12d2-4e92-a63b-4b442bed056e.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-PortMirroringConfigurationByPolicy-template.yml,0,ddb99ebb-12d2-4e92-a63b-4b442bed056e.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=504924bc-fad2-4165-9729-61dc20768766 EntryTimestamp=2022-04-26T06:23:21.244Z InvocationID=19f98b18-b2f9-42b2-b754-2538b377727f lock resource with id ddb99ebb-12d2-4e92-a63b-4b442bed056e 2022-04-26T06:23:24.864Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Port Mirroring Configuration By Policy" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "2e39e2c1-9834-4ef8-acfa-77116ce76418" INVARIANT_UUID = "0b6c81f5-0b38-4bec-bf40-f7a494ca1ca3" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ddb99ebb-12d2-4e92-a63b-4b442bed056e.assettoscacsar" CURR_ARTIFACT_UUID = "ddb99ebb-12d2-4e92-a63b-4b442bed056e.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-PortMirroringConfigurationByPolicy-csar.csar,0,ddb99ebb-12d2-4e92-a63b-4b442bed056e.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=3620 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4be87991-7f91-41d8-99a1-53355a73f737 EntryTimestamp=2022-04-26T06:23:21.244Z InvocationID=19f98b18-b2f9-42b2-b754-2538b377727f LogTimestamp=2022-04-26T06:23:24.864507Z ResponseDesc=Created Exiting. 2022-04-26T06:23:24.865Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:23:24.877Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1c30c324-77a5-42e6-8201-5799a93a4e7f InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:24.877158Z InvocationID=e6fe8b42-fb8c-42e8-9862-78f6f9e8b751 LogTimestamp=2022-04-26T06:23:24.877235Z Entering 2022-04-26T06:23:24.877Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1c30c324-77a5-42e6-8201-5799a93a4e7f InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:24.877158Z InvocationID=e6fe8b42-fb8c-42e8-9862-78f6f9e8b751 LogTimestamp=2022-04-26T06:23:24.877235Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:23:25.546Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5eb22dd-aa46-4728-ae7d-5b465be54689 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:23:25.546Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5eb22dd-aa46-4728-ae7d-5b465be54689 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4045efd5 2022-04-26T06:23:25.546Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5eb22dd-aa46-4728-ae7d-5b465be54689 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:23:25.546Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5eb22dd-aa46-4728-ae7d-5b465be54689 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:23:25.663Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:24.878Z InvocationID=b7861b2a-c4ab-4a57-bcc8-330b34c249e3 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:25.663Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:24.878Z InvocationID=b7861b2a-c4ab-4a57-bcc8-330b34c249e3 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:25.663Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:24.878Z InvocationID=b7861b2a-c4ab-4a57-bcc8-330b34c249e3 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:25.663Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:24.878Z InvocationID=b7861b2a-c4ab-4a57-bcc8-330b34c249e3 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:25.741Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:24.878Z InvocationID=b7861b2a-c4ab-4a57-bcc8-330b34c249e3 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name ForwardingPath 2022-04-26T06:23:26.062Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Forwarding Path" RESOURCE_TYPE = "ABSTRACT" SERVICE_INSTANCE_ID = "26eafc88-5dec-4643-bde8-b2c55f57a7e5" INVARIANT_UUID = "ae0ae7eb-a130-4999-a2dc-c67d08e39879" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.nodes.ForwardingPath" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=26eafc88-5dec-4643-bde8-b2c55f57a7e5 EntryTimestamp=2022-04-26T06:23:24.878Z InvocationID=b7861b2a-c4ab-4a57-bcc8-330b34c249e3 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id ed314100-b6f3-45ab-88a5-fc946734e665 2022-04-26T06:23:26.574Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f5eb22dd-aa46-4728-ae7d-5b465be54689 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:23:26.574Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f5eb22dd-aa46-4728-ae7d-5b465be54689 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:23:27.360Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Forwarding Path" RESOURCE_TYPE = "ABSTRACT" SERVICE_INSTANCE_ID = "26eafc88-5dec-4643-bde8-b2c55f57a7e5" INVARIANT_UUID = "ae0ae7eb-a130-4999-a2dc-c67d08e39879" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ed314100-b6f3-45ab-88a5-fc946734e665.assettoscatemplate" CURR_ARTIFACT_UUID = "ed314100-b6f3-45ab-88a5-fc946734e665.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ForwardingPath-template.yml,0,ed314100-b6f3-45ab-88a5-fc946734e665.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=2482 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=91469e59-6c79-472d-ac68-56f4d14bd7f8 EntryTimestamp=2022-04-26T06:23:24.878Z InvocationID=b7861b2a-c4ab-4a57-bcc8-330b34c249e3 LogTimestamp=2022-04-26T06:23:27.360610Z ResponseDesc=Created Exiting. 2022-04-26T06:23:27.361Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:23:27.373Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=937bf9dd-a2aa-4361-b5c6-e6efd51794ef InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:27.373074Z InvocationID=b9581aa0-3ded-451a-81e6-e96da752bbd9 LogTimestamp=2022-04-26T06:23:27.373186Z Entering 2022-04-26T06:23:27.374Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=937bf9dd-a2aa-4361-b5c6-e6efd51794ef InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:27.373074Z InvocationID=b9581aa0-3ded-451a-81e6-e96da752bbd9 LogTimestamp=2022-04-26T06:23:27.373186Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:23:28.243Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:27.374Z InvocationID=20798ed0-6548-4249-aec5-f7a55281198c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:28.243Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:27.374Z InvocationID=20798ed0-6548-4249-aec5-f7a55281198c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:28.243Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:27.374Z InvocationID=20798ed0-6548-4249-aec5-f7a55281198c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:28.243Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:27.374Z InvocationID=20798ed0-6548-4249-aec5-f7a55281198c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:28.244Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:27.374Z InvocationID=20798ed0-6548-4249-aec5-f7a55281198c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Configuration 2022-04-26T06:23:28.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c51c6aa6-459d-4f7d-9b58-3010a13e6c54 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:23:28.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c51c6aa6-459d-4f7d-9b58-3010a13e6c54 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@48d50035 2022-04-26T06:23:28.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c51c6aa6-459d-4f7d-9b58-3010a13e6c54 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:23:28.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c51c6aa6-459d-4f7d-9b58-3010a13e6c54 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:23:28.756Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Configuration" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "b8e0dcd6-6a79-4959-bd58-a9a4045e73e2" INVARIANT_UUID = "a3788396-d47f-4c19-874c-de066e77853e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.nodes.Configuration" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b8e0dcd6-6a79-4959-bd58-a9a4045e73e2 EntryTimestamp=2022-04-26T06:23:27.374Z InvocationID=20798ed0-6548-4249-aec5-f7a55281198c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id bf979ed6-b5af-44ca-a725-5b48c6a6d9a6 2022-04-26T06:23:29.673Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c51c6aa6-459d-4f7d-9b58-3010a13e6c54 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:23:29.673Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c51c6aa6-459d-4f7d-9b58-3010a13e6c54 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:23:30.933Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Configuration" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "b8e0dcd6-6a79-4959-bd58-a9a4045e73e2" INVARIANT_UUID = "a3788396-d47f-4c19-874c-de066e77853e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "bf979ed6-b5af-44ca-a725-5b48c6a6d9a6.assettoscatemplate" CURR_ARTIFACT_UUID = "bf979ed6-b5af-44ca-a725-5b48c6a6d9a6.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Configuration-template.yml,0,bf979ed6-b5af-44ca-a725-5b48c6a6d9a6.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=3559 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d7ec074d-a807-45d0-9f15-785f53cd6225 EntryTimestamp=2022-04-26T06:23:27.374Z InvocationID=20798ed0-6548-4249-aec5-f7a55281198c LogTimestamp=2022-04-26T06:23:30.933583Z ResponseDesc=Created Exiting. 2022-04-26T06:23:30.935Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:23:30.945Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=265a93bd-ce5f-4cf0-860d-aec9ae7ad43c InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:30.945021Z InvocationID=31b5dafd-0f19-485d-bad7-095f210bb070 LogTimestamp=2022-04-26T06:23:30.945096Z Entering 2022-04-26T06:23:30.945Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=265a93bd-ce5f-4cf0-860d-aec9ae7ad43c InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:30.945021Z InvocationID=31b5dafd-0f19-485d-bad7-095f210bb070 LogTimestamp=2022-04-26T06:23:30.945096Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:23:31.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d1a17a0e-fd52-444b-8e32-dfb94f46a85e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:23:31.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d1a17a0e-fd52-444b-8e32-dfb94f46a85e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8593cb6 2022-04-26T06:23:31.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d1a17a0e-fd52-444b-8e32-dfb94f46a85e PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:23:31.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d1a17a0e-fd52-444b-8e32-dfb94f46a85e PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:23:32.557Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:30.945Z InvocationID=cceabef3-6066-4883-9cf6-5d494fefcbf0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:32.557Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:30.945Z InvocationID=cceabef3-6066-4883-9cf6-5d494fefcbf0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:32.557Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:30.945Z InvocationID=cceabef3-6066-4883-9cf6-5d494fefcbf0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:32.557Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:30.945Z InvocationID=cceabef3-6066-4883-9cf6-5d494fefcbf0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:32.591Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=d1a17a0e-fd52-444b-8e32-dfb94f46a85e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:23:32.591Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=d1a17a0e-fd52-444b-8e32-dfb94f46a85e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:23:32.733Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:30.945Z InvocationID=cceabef3-6066-4883-9cf6-5d494fefcbf0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name VrfObjectConfiguration 2022-04-26T06:23:33.345Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VRF Object Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "d8a077b3-aebd-4040-b1b5-09db816fb14e" INVARIANT_UUID = "4aa076aa-e4be-4718-a5df-034b10f2985c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.nodes.VRFObject" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d8a077b3-aebd-4040-b1b5-09db816fb14e EntryTimestamp=2022-04-26T06:23:30.945Z InvocationID=cceabef3-6066-4883-9cf6-5d494fefcbf0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 1087eae9-4715-4144-828a-75800b459669 2022-04-26T06:23:34.436Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=498481ea-8437-41c2-b662-c027b8177b3f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:23:34.436Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=498481ea-8437-41c2-b662-c027b8177b3f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@54a75bdc 2022-04-26T06:23:34.436Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=498481ea-8437-41c2-b662-c027b8177b3f PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:23:34.437Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=498481ea-8437-41c2-b662-c027b8177b3f PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:23:35.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:23:35.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:23:35.467Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=498481ea-8437-41c2-b662-c027b8177b3f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:23:35.468Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=498481ea-8437-41c2-b662-c027b8177b3f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:23:35.539Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Object Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "d8a077b3-aebd-4040-b1b5-09db816fb14e" INVARIANT_UUID = "4aa076aa-e4be-4718-a5df-034b10f2985c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1087eae9-4715-4144-828a-75800b459669.assettoscatemplate" CURR_ARTIFACT_UUID = "1087eae9-4715-4144-828a-75800b459669.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfObjectConfiguration-template.yml,0,1087eae9-4715-4144-828a-75800b459669.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=549752d8-f41b-44f9-b092-3a0fd14986a5 EntryTimestamp=2022-04-26T06:23:30.945Z InvocationID=cceabef3-6066-4883-9cf6-5d494fefcbf0 PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Starting to create Csar for component VRF Object Configuration 2022-04-26T06:23:35.792Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Object Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "d8a077b3-aebd-4040-b1b5-09db816fb14e" INVARIANT_UUID = "4aa076aa-e4be-4718-a5df-034b10f2985c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1087eae9-4715-4144-828a-75800b459669.assettoscatemplate" CURR_ARTIFACT_UUID = "1087eae9-4715-4144-828a-75800b459669.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfObjectConfiguration-template.yml,0,1087eae9-4715-4144-828a-75800b459669.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=549752d8-f41b-44f9-b092-3a0fd14986a5 EntryTimestamp=2022-04-26T06:23:30.945Z InvocationID=cceabef3-6066-4883-9cf6-5d494fefcbf0 Ended create Csar for component VRF Object Configuration 2022-04-26T06:23:35.792Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Object Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "d8a077b3-aebd-4040-b1b5-09db816fb14e" INVARIANT_UUID = "4aa076aa-e4be-4718-a5df-034b10f2985c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1087eae9-4715-4144-828a-75800b459669.assettoscatemplate" CURR_ARTIFACT_UUID = "1087eae9-4715-4144-828a-75800b459669.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfObjectConfiguration-template.yml,0,1087eae9-4715-4144-828a-75800b459669.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=549752d8-f41b-44f9-b092-3a0fd14986a5 EntryTimestamp=2022-04-26T06:23:30.945Z InvocationID=cceabef3-6066-4883-9cf6-5d494fefcbf0 lock resource with id 1087eae9-4715-4144-828a-75800b459669 2022-04-26T06:23:36.651Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Object Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "d8a077b3-aebd-4040-b1b5-09db816fb14e" INVARIANT_UUID = "4aa076aa-e4be-4718-a5df-034b10f2985c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1087eae9-4715-4144-828a-75800b459669.assettoscacsar" CURR_ARTIFACT_UUID = "1087eae9-4715-4144-828a-75800b459669.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VrfObjectConfiguration-csar.csar,0,1087eae9-4715-4144-828a-75800b459669.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5706 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f9f2a4c9-fe79-4d3f-9ae0-da2c10cc45e5 EntryTimestamp=2022-04-26T06:23:30.945Z InvocationID=cceabef3-6066-4883-9cf6-5d494fefcbf0 LogTimestamp=2022-04-26T06:23:36.651427Z ResponseDesc=Created Exiting. 2022-04-26T06:23:36.652Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:23:36.669Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6444ca70-b612-46fe-bfa9-05e8b07a8faf InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:36.668677Z InvocationID=13086f56-4654-4323-a43b-516f2c202ca8 LogTimestamp=2022-04-26T06:23:36.668765Z Entering 2022-04-26T06:23:36.669Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6444ca70-b612-46fe-bfa9-05e8b07a8faf InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:36.668677Z InvocationID=13086f56-4654-4323-a43b-516f2c202ca8 LogTimestamp=2022-04-26T06:23:36.668765Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:23:37.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe703812-71dc-4989-aa33-47fbb28480e5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:23:37.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe703812-71dc-4989-aa33-47fbb28480e5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@18bc656b 2022-04-26T06:23:37.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe703812-71dc-4989-aa33-47fbb28480e5 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:23:37.639Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe703812-71dc-4989-aa33-47fbb28480e5 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:23:38.041Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:36.669Z InvocationID=06b9e37f-330a-422b-abeb-9ceeb59265c2 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:38.043Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:36.669Z InvocationID=06b9e37f-330a-422b-abeb-9ceeb59265c2 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:38.045Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:36.669Z InvocationID=06b9e37f-330a-422b-abeb-9ceeb59265c2 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:38.045Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:36.669Z InvocationID=06b9e37f-330a-422b-abeb-9ceeb59265c2 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:38.130Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:36.669Z InvocationID=06b9e37f-330a-422b-abeb-9ceeb59265c2 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Extvirtualmachineinterfacecp 2022-04-26T06:23:38.546Z [qtp2085002312-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "extVirtualMachineInterfaceCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "a3a21349-6f3d-4f13-8ec3-a45c5dcd2289" INVARIANT_UUID = "07308fbe-250c-4a52-9b94-39e7881f5a6d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.v2.extVirtualMachineInterfaceCP" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a3a21349-6f3d-4f13-8ec3-a45c5dcd2289 EntryTimestamp=2022-04-26T06:23:36.669Z InvocationID=06b9e37f-330a-422b-abeb-9ceeb59265c2 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:23:38.547Z [qtp2085002312-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "extVirtualMachineInterfaceCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "a3a21349-6f3d-4f13-8ec3-a45c5dcd2289" INVARIANT_UUID = "07308fbe-250c-4a52-9b94-39e7881f5a6d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.v2.extVirtualMachineInterfaceCP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a3a21349-6f3d-4f13-8ec3-a45c5dcd2289 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:36.669Z InvocationID=06b9e37f-330a-422b-abeb-9ceeb59265c2 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:23:38.673Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=fe703812-71dc-4989-aa33-47fbb28480e5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:23:38.673Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=fe703812-71dc-4989-aa33-47fbb28480e5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:23:38.951Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "extVirtualMachineInterfaceCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "a3a21349-6f3d-4f13-8ec3-a45c5dcd2289" INVARIANT_UUID = "07308fbe-250c-4a52-9b94-39e7881f5a6d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.v2.extVirtualMachineInterfaceCP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a3a21349-6f3d-4f13-8ec3-a45c5dcd2289 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:36.669Z InvocationID=06b9e37f-330a-422b-abeb-9ceeb59265c2 ErrorCode=500 lock resource with id 544519da-98dc-45f6-af3c-f73ea1675cef 2022-04-26T06:23:40.343Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e351a00-faa7-4669-8387-05e595bd83a8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:23:40.343Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e351a00-faa7-4669-8387-05e595bd83a8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5b9b1650 2022-04-26T06:23:40.343Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e351a00-faa7-4669-8387-05e595bd83a8 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:23:40.343Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e351a00-faa7-4669-8387-05e595bd83a8 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:23:40.643Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "extVirtualMachineInterfaceCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "a3a21349-6f3d-4f13-8ec3-a45c5dcd2289" INVARIANT_UUID = "07308fbe-250c-4a52-9b94-39e7881f5a6d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "544519da-98dc-45f6-af3c-f73ea1675cef.assettoscatemplate" CURR_ARTIFACT_UUID = "544519da-98dc-45f6-af3c-f73ea1675cef.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extvirtualmachineinterfacecp-template.yml,0,544519da-98dc-45f6-af3c-f73ea1675cef.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d837a7eb-17fa-4d61-a584-cf7b26d0f922 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:36.669Z InvocationID=06b9e37f-330a-422b-abeb-9ceeb59265c2 ErrorCode=500 Starting to create Csar for component extVirtualMachineInterfaceCP 2022-04-26T06:23:40.802Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "extVirtualMachineInterfaceCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "a3a21349-6f3d-4f13-8ec3-a45c5dcd2289" INVARIANT_UUID = "07308fbe-250c-4a52-9b94-39e7881f5a6d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "544519da-98dc-45f6-af3c-f73ea1675cef.assettoscatemplate" CURR_ARTIFACT_UUID = "544519da-98dc-45f6-af3c-f73ea1675cef.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extvirtualmachineinterfacecp-template.yml,0,544519da-98dc-45f6-af3c-f73ea1675cef.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d837a7eb-17fa-4d61-a584-cf7b26d0f922 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:36.669Z InvocationID=06b9e37f-330a-422b-abeb-9ceeb59265c2 ErrorCode=500 Ended create Csar for component extVirtualMachineInterfaceCP 2022-04-26T06:23:40.803Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "extVirtualMachineInterfaceCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "a3a21349-6f3d-4f13-8ec3-a45c5dcd2289" INVARIANT_UUID = "07308fbe-250c-4a52-9b94-39e7881f5a6d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "544519da-98dc-45f6-af3c-f73ea1675cef.assettoscatemplate" CURR_ARTIFACT_UUID = "544519da-98dc-45f6-af3c-f73ea1675cef.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extvirtualmachineinterfacecp-template.yml,0,544519da-98dc-45f6-af3c-f73ea1675cef.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d837a7eb-17fa-4d61-a584-cf7b26d0f922 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:36.669Z InvocationID=06b9e37f-330a-422b-abeb-9ceeb59265c2 ErrorCode=500 lock resource with id 544519da-98dc-45f6-af3c-f73ea1675cef 2022-04-26T06:23:41.392Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=1e351a00-faa7-4669-8387-05e595bd83a8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:23:41.393Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=1e351a00-faa7-4669-8387-05e595bd83a8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:23:41.499Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "extVirtualMachineInterfaceCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "a3a21349-6f3d-4f13-8ec3-a45c5dcd2289" INVARIANT_UUID = "07308fbe-250c-4a52-9b94-39e7881f5a6d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "544519da-98dc-45f6-af3c-f73ea1675cef.assettoscacsar" CURR_ARTIFACT_UUID = "544519da-98dc-45f6-af3c-f73ea1675cef.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Extvirtualmachineinterfacecp-csar.csar,0,544519da-98dc-45f6-af3c-f73ea1675cef.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=4830 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1f5e1125-242f-4321-b309-12357e345481 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:36.669Z InvocationID=06b9e37f-330a-422b-abeb-9ceeb59265c2 ErrorCode=500 LogTimestamp=2022-04-26T06:23:41.499033Z ResponseDesc=Created Exiting. 2022-04-26T06:23:41.501Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:23:41.521Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1f80a505-9a53-4188-8422-abe1b17fc71c InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:41.520508Z InvocationID=51b0782b-65a7-43b1-a7cf-8dece74e97da LogTimestamp=2022-04-26T06:23:41.520716Z Entering 2022-04-26T06:23:41.521Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1f80a505-9a53-4188-8422-abe1b17fc71c InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:41.520508Z InvocationID=51b0782b-65a7-43b1-a7cf-8dece74e97da LogTimestamp=2022-04-26T06:23:41.520716Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:23:41.658Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:41.521Z InvocationID=773ba684-431f-4d86-864e-95b2f3505f90 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:41.658Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:41.521Z InvocationID=773ba684-431f-4d86-864e-95b2f3505f90 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:41.658Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:41.521Z InvocationID=773ba684-431f-4d86-864e-95b2f3505f90 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:41.659Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:41.521Z InvocationID=773ba684-431f-4d86-864e-95b2f3505f90 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:41.687Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:41.521Z InvocationID=773ba684-431f-4d86-864e-95b2f3505f90 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name VlanNetworkReceptorConfiguration 2022-04-26T06:23:41.786Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VLAN Network Receptor Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "776ebd30-b104-4300-b0df-32fd305e9512" INVARIANT_UUID = "d6a3dff2-4902-4dd5-af88-10da3dced8f5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.nodes.VLANNetworkReceptor" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=776ebd30-b104-4300-b0df-32fd305e9512 EntryTimestamp=2022-04-26T06:23:41.521Z InvocationID=773ba684-431f-4d86-864e-95b2f3505f90 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 676194b2-db33-44c4-92c3-9fde9aa40e73 2022-04-26T06:23:42.450Z [qtp2085002312-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VLAN Network Receptor Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "776ebd30-b104-4300-b0df-32fd305e9512" INVARIANT_UUID = "d6a3dff2-4902-4dd5-af88-10da3dced8f5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "676194b2-db33-44c4-92c3-9fde9aa40e73.assettoscatemplate" CURR_ARTIFACT_UUID = "676194b2-db33-44c4-92c3-9fde9aa40e73.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VlanNetworkReceptorConfiguration-template.yml,0,676194b2-db33-44c4-92c3-9fde9aa40e73.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b74fbd5b-619d-42f0-91c8-45019de76b6d EntryTimestamp=2022-04-26T06:23:41.521Z InvocationID=773ba684-431f-4d86-864e-95b2f3505f90 PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Starting to create Csar for component VLAN Network Receptor Configuration 2022-04-26T06:23:42.624Z [qtp2085002312-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VLAN Network Receptor Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "776ebd30-b104-4300-b0df-32fd305e9512" INVARIANT_UUID = "d6a3dff2-4902-4dd5-af88-10da3dced8f5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "676194b2-db33-44c4-92c3-9fde9aa40e73.assettoscatemplate" CURR_ARTIFACT_UUID = "676194b2-db33-44c4-92c3-9fde9aa40e73.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VlanNetworkReceptorConfiguration-template.yml,0,676194b2-db33-44c4-92c3-9fde9aa40e73.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b74fbd5b-619d-42f0-91c8-45019de76b6d EntryTimestamp=2022-04-26T06:23:41.521Z InvocationID=773ba684-431f-4d86-864e-95b2f3505f90 Ended create Csar for component VLAN Network Receptor Configuration 2022-04-26T06:23:42.625Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VLAN Network Receptor Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "776ebd30-b104-4300-b0df-32fd305e9512" INVARIANT_UUID = "d6a3dff2-4902-4dd5-af88-10da3dced8f5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "676194b2-db33-44c4-92c3-9fde9aa40e73.assettoscatemplate" CURR_ARTIFACT_UUID = "676194b2-db33-44c4-92c3-9fde9aa40e73.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VlanNetworkReceptorConfiguration-template.yml,0,676194b2-db33-44c4-92c3-9fde9aa40e73.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b74fbd5b-619d-42f0-91c8-45019de76b6d EntryTimestamp=2022-04-26T06:23:41.521Z InvocationID=773ba684-431f-4d86-864e-95b2f3505f90 lock resource with id 676194b2-db33-44c4-92c3-9fde9aa40e73 2022-04-26T06:23:42.802Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=646884f2-55c1-4b28-8a61-cbcc70c819a8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:23:42.802Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=646884f2-55c1-4b28-8a61-cbcc70c819a8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@73961285 2022-04-26T06:23:42.803Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=646884f2-55c1-4b28-8a61-cbcc70c819a8 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:23:42.803Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=646884f2-55c1-4b28-8a61-cbcc70c819a8 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:23:43.188Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VLAN Network Receptor Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "776ebd30-b104-4300-b0df-32fd305e9512" INVARIANT_UUID = "d6a3dff2-4902-4dd5-af88-10da3dced8f5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "676194b2-db33-44c4-92c3-9fde9aa40e73.assettoscacsar" CURR_ARTIFACT_UUID = "676194b2-db33-44c4-92c3-9fde9aa40e73.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VlanNetworkReceptorConfiguration-csar.csar,0,676194b2-db33-44c4-92c3-9fde9aa40e73.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1666 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4d7eeb0b-9765-484b-b1c8-30ee1e31e550 EntryTimestamp=2022-04-26T06:23:41.521Z InvocationID=773ba684-431f-4d86-864e-95b2f3505f90 LogTimestamp=2022-04-26T06:23:43.187582Z ResponseDesc=Created Exiting. 2022-04-26T06:23:43.188Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:23:43.205Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6957ad12-d252-4129-85ec-68eebd2738d9 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:43.204642Z InvocationID=c88b95f3-d937-474d-915a-63fe7b7da824 LogTimestamp=2022-04-26T06:23:43.204736Z Entering 2022-04-26T06:23:43.205Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6957ad12-d252-4129-85ec-68eebd2738d9 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:43.204642Z InvocationID=c88b95f3-d937-474d-915a-63fe7b7da824 LogTimestamp=2022-04-26T06:23:43.204736Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:23:43.311Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:43.205Z InvocationID=725e7fc1-e135-47cf-b766-81f4f20e5de1 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:43.312Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:43.205Z InvocationID=725e7fc1-e135-47cf-b766-81f4f20e5de1 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:43.312Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:43.205Z InvocationID=725e7fc1-e135-47cf-b766-81f4f20e5de1 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:43.312Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:43.205Z InvocationID=725e7fc1-e135-47cf-b766-81f4f20e5de1 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:43.317Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:43.205Z InvocationID=725e7fc1-e135-47cf-b766-81f4f20e5de1 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name VrfEntryConfiguration 2022-04-26T06:23:43.417Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "e3f0218f-095c-4d9b-9449-29873ea6ef5d" INVARIANT_UUID = "20765435-3242-415e-ba22-959aa7b2005a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.nodes.VRFEntry" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e3f0218f-095c-4d9b-9449-29873ea6ef5d EntryTimestamp=2022-04-26T06:23:43.205Z InvocationID=725e7fc1-e135-47cf-b766-81f4f20e5de1 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 41e84520-74d2-48d6-b40e-71b926d81762 2022-04-26T06:23:43.863Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=646884f2-55c1-4b28-8a61-cbcc70c819a8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:23:43.866Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=646884f2-55c1-4b28-8a61-cbcc70c819a8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:23:44.050Z [qtp2085002312-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "e3f0218f-095c-4d9b-9449-29873ea6ef5d" INVARIANT_UUID = "20765435-3242-415e-ba22-959aa7b2005a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "41e84520-74d2-48d6-b40e-71b926d81762.assettoscatemplate" CURR_ARTIFACT_UUID = "41e84520-74d2-48d6-b40e-71b926d81762.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,41e84520-74d2-48d6-b40e-71b926d81762.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d963f88f-6f6c-48ac-8df9-400c0a7986b1 EntryTimestamp=2022-04-26T06:23:43.205Z InvocationID=725e7fc1-e135-47cf-b766-81f4f20e5de1 PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Starting to create Csar for component VRF Entry Configuration 2022-04-26T06:23:44.305Z [qtp2085002312-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "e3f0218f-095c-4d9b-9449-29873ea6ef5d" INVARIANT_UUID = "20765435-3242-415e-ba22-959aa7b2005a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "41e84520-74d2-48d6-b40e-71b926d81762.assettoscatemplate" CURR_ARTIFACT_UUID = "41e84520-74d2-48d6-b40e-71b926d81762.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,41e84520-74d2-48d6-b40e-71b926d81762.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d963f88f-6f6c-48ac-8df9-400c0a7986b1 EntryTimestamp=2022-04-26T06:23:43.205Z InvocationID=725e7fc1-e135-47cf-b766-81f4f20e5de1 Ended create Csar for component VRF Entry Configuration 2022-04-26T06:23:44.306Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "e3f0218f-095c-4d9b-9449-29873ea6ef5d" INVARIANT_UUID = "20765435-3242-415e-ba22-959aa7b2005a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "41e84520-74d2-48d6-b40e-71b926d81762.assettoscatemplate" CURR_ARTIFACT_UUID = "41e84520-74d2-48d6-b40e-71b926d81762.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,41e84520-74d2-48d6-b40e-71b926d81762.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d963f88f-6f6c-48ac-8df9-400c0a7986b1 EntryTimestamp=2022-04-26T06:23:43.205Z InvocationID=725e7fc1-e135-47cf-b766-81f4f20e5de1 lock resource with id 41e84520-74d2-48d6-b40e-71b926d81762 2022-04-26T06:23:44.877Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "e3f0218f-095c-4d9b-9449-29873ea6ef5d" INVARIANT_UUID = "20765435-3242-415e-ba22-959aa7b2005a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "41e84520-74d2-48d6-b40e-71b926d81762.assettoscacsar" CURR_ARTIFACT_UUID = "41e84520-74d2-48d6-b40e-71b926d81762.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VrfEntryConfiguration-csar.csar,0,41e84520-74d2-48d6-b40e-71b926d81762.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1672 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=56cbe5d6-9990-4a1f-b56b-9f0e3d1767d2 EntryTimestamp=2022-04-26T06:23:43.205Z InvocationID=725e7fc1-e135-47cf-b766-81f4f20e5de1 LogTimestamp=2022-04-26T06:23:44.877210Z ResponseDesc=Created Exiting. 2022-04-26T06:23:44.878Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:23:44.892Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=718521f6-edf2-4e23-a2a5-ff38e3287cbc InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:44.892082Z InvocationID=ca0d5620-520d-4bb6-9565-8003c1b31e0e LogTimestamp=2022-04-26T06:23:44.892182Z Entering 2022-04-26T06:23:44.892Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=718521f6-edf2-4e23-a2a5-ff38e3287cbc InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:44.892082Z InvocationID=ca0d5620-520d-4bb6-9565-8003c1b31e0e LogTimestamp=2022-04-26T06:23:44.892182Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:23:45.054Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:44.892Z InvocationID=08b82f52-1635-4792-b10c-1b414d24a528 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:45.054Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:44.892Z InvocationID=08b82f52-1635-4792-b10c-1b414d24a528 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:45.055Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:44.892Z InvocationID=08b82f52-1635-4792-b10c-1b414d24a528 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:45.055Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:44.892Z InvocationID=08b82f52-1635-4792-b10c-1b414d24a528 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:45.063Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:44.892Z InvocationID=08b82f52-1635-4792-b10c-1b414d24a528 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Subinterfacev2 2022-04-26T06:23:45.114Z [qtp2085002312-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "subInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "c50e22f6-ff8e-4cdf-a414-f6df9e396bec" INVARIANT_UUID = "f7ef4f75-a8e3-4d63-8585-440c14f3afef" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.network.v2.SubInterface" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c50e22f6-ff8e-4cdf-a414-f6df9e396bec EntryTimestamp=2022-04-26T06:23:44.892Z InvocationID=08b82f52-1635-4792-b10c-1b414d24a528 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:23:45.115Z [qtp2085002312-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "subInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "c50e22f6-ff8e-4cdf-a414-f6df9e396bec" INVARIANT_UUID = "f7ef4f75-a8e3-4d63-8585-440c14f3afef" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.network.v2.SubInterface" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c50e22f6-ff8e-4cdf-a414-f6df9e396bec TargetServiceName=null EntryTimestamp=2022-04-26T06:23:44.892Z InvocationID=08b82f52-1635-4792-b10c-1b414d24a528 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:23:45.159Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "subInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "c50e22f6-ff8e-4cdf-a414-f6df9e396bec" INVARIANT_UUID = "f7ef4f75-a8e3-4d63-8585-440c14f3afef" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.network.v2.SubInterface" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c50e22f6-ff8e-4cdf-a414-f6df9e396bec TargetServiceName=null EntryTimestamp=2022-04-26T06:23:44.892Z InvocationID=08b82f52-1635-4792-b10c-1b414d24a528 ErrorCode=500 lock resource with id 07d75ae5-49d5-4a82-a795-233e36345d17 2022-04-26T06:23:45.939Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b1ee93b-c460-45ac-b29c-cd1af865a7e5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:23:45.939Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b1ee93b-c460-45ac-b29c-cd1af865a7e5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@23f7ddde 2022-04-26T06:23:45.939Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b1ee93b-c460-45ac-b29c-cd1af865a7e5 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:23:45.940Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b1ee93b-c460-45ac-b29c-cd1af865a7e5 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:23:45.958Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "subInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "c50e22f6-ff8e-4cdf-a414-f6df9e396bec" INVARIANT_UUID = "f7ef4f75-a8e3-4d63-8585-440c14f3afef" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "07d75ae5-49d5-4a82-a795-233e36345d17.assettoscatemplate" CURR_ARTIFACT_UUID = "07d75ae5-49d5-4a82-a795-233e36345d17.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Subinterfacev2-template.yml,0,07d75ae5-49d5-4a82-a795-233e36345d17.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=61a626b5-77a1-4fd0-a75c-e845584c5165 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:44.892Z InvocationID=08b82f52-1635-4792-b10c-1b414d24a528 ErrorCode=500 Starting to create Csar for component subInterfaceV2 2022-04-26T06:23:46.158Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "subInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "c50e22f6-ff8e-4cdf-a414-f6df9e396bec" INVARIANT_UUID = "f7ef4f75-a8e3-4d63-8585-440c14f3afef" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "07d75ae5-49d5-4a82-a795-233e36345d17.assettoscatemplate" CURR_ARTIFACT_UUID = "07d75ae5-49d5-4a82-a795-233e36345d17.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Subinterfacev2-template.yml,0,07d75ae5-49d5-4a82-a795-233e36345d17.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=61a626b5-77a1-4fd0-a75c-e845584c5165 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:44.892Z InvocationID=08b82f52-1635-4792-b10c-1b414d24a528 ErrorCode=500 Ended create Csar for component subInterfaceV2 2022-04-26T06:23:46.159Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "subInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "c50e22f6-ff8e-4cdf-a414-f6df9e396bec" INVARIANT_UUID = "f7ef4f75-a8e3-4d63-8585-440c14f3afef" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "07d75ae5-49d5-4a82-a795-233e36345d17.assettoscatemplate" CURR_ARTIFACT_UUID = "07d75ae5-49d5-4a82-a795-233e36345d17.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Subinterfacev2-template.yml,0,07d75ae5-49d5-4a82-a795-233e36345d17.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=61a626b5-77a1-4fd0-a75c-e845584c5165 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:44.892Z InvocationID=08b82f52-1635-4792-b10c-1b414d24a528 ErrorCode=500 lock resource with id 07d75ae5-49d5-4a82-a795-233e36345d17 2022-04-26T06:23:46.782Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "subInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "c50e22f6-ff8e-4cdf-a414-f6df9e396bec" INVARIANT_UUID = "f7ef4f75-a8e3-4d63-8585-440c14f3afef" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "07d75ae5-49d5-4a82-a795-233e36345d17.assettoscacsar" CURR_ARTIFACT_UUID = "07d75ae5-49d5-4a82-a795-233e36345d17.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Subinterfacev2-csar.csar,0,07d75ae5-49d5-4a82-a795-233e36345d17.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1889 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=61b26ae9-b1a9-4eba-a947-fa173a131993 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:44.892Z InvocationID=08b82f52-1635-4792-b10c-1b414d24a528 ErrorCode=500 LogTimestamp=2022-04-26T06:23:46.781841Z ResponseDesc=Created Exiting. 2022-04-26T06:23:46.782Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:23:46.796Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=95d7d4f2-8cbb-43cd-92aa-f2c72bc7603d InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:46.796079Z InvocationID=56de8daa-c75e-4d2d-aa32-ed0e5f1cdd25 LogTimestamp=2022-04-26T06:23:46.796158Z Entering 2022-04-26T06:23:46.796Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=95d7d4f2-8cbb-43cd-92aa-f2c72bc7603d InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:46.796079Z InvocationID=56de8daa-c75e-4d2d-aa32-ed0e5f1cdd25 LogTimestamp=2022-04-26T06:23:46.796158Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:23:46.943Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:46.797Z InvocationID=ebb75740-8071-412b-93cc-ee44c3166407 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:46.944Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:46.797Z InvocationID=ebb75740-8071-412b-93cc-ee44c3166407 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:46.944Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:46.797Z InvocationID=ebb75740-8071-412b-93cc-ee44c3166407 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:46.944Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:46.797Z InvocationID=ebb75740-8071-412b-93cc-ee44c3166407 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:46.948Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:46.797Z InvocationID=ebb75740-8071-412b-93cc-ee44c3166407 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Contrailv2vlansubinterfacev2 2022-04-26T06:23:46.957Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=5b1ee93b-c460-45ac-b29c-cd1af865a7e5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:23:46.958Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=5b1ee93b-c460-45ac-b29c-cd1af865a7e5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:23:46.996Z [qtp2085002312-13] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "contrailV2VLANSubInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "ad83f439-9339-4405-a180-b96e227c9a4c" INVARIANT_UUID = "9ec3cc86-bab1-4dd9-be5f-98e4f2ef90db" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.network.v2.contrailV2.VLANSubInterface" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ad83f439-9339-4405-a180-b96e227c9a4c EntryTimestamp=2022-04-26T06:23:46.797Z InvocationID=ebb75740-8071-412b-93cc-ee44c3166407 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:23:46.997Z [qtp2085002312-13] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "contrailV2VLANSubInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "ad83f439-9339-4405-a180-b96e227c9a4c" INVARIANT_UUID = "9ec3cc86-bab1-4dd9-be5f-98e4f2ef90db" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.network.v2.contrailV2.VLANSubInterface" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ad83f439-9339-4405-a180-b96e227c9a4c TargetServiceName=null EntryTimestamp=2022-04-26T06:23:46.797Z InvocationID=ebb75740-8071-412b-93cc-ee44c3166407 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:23:47.034Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "contrailV2VLANSubInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "ad83f439-9339-4405-a180-b96e227c9a4c" INVARIANT_UUID = "9ec3cc86-bab1-4dd9-be5f-98e4f2ef90db" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.network.v2.contrailV2.VLANSubInterface" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ad83f439-9339-4405-a180-b96e227c9a4c TargetServiceName=null EntryTimestamp=2022-04-26T06:23:46.797Z InvocationID=ebb75740-8071-412b-93cc-ee44c3166407 ErrorCode=500 lock resource with id 13f8cc99-9ccf-4de6-ad3e-533f2b9886b9 2022-04-26T06:23:47.647Z [qtp2085002312-13] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "contrailV2VLANSubInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "ad83f439-9339-4405-a180-b96e227c9a4c" INVARIANT_UUID = "9ec3cc86-bab1-4dd9-be5f-98e4f2ef90db" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "13f8cc99-9ccf-4de6-ad3e-533f2b9886b9.assettoscatemplate" CURR_ARTIFACT_UUID = "13f8cc99-9ccf-4de6-ad3e-533f2b9886b9.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2vlansubinterfacev2-template.yml,0,13f8cc99-9ccf-4de6-ad3e-533f2b9886b9.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d9526003-3e5a-49c7-afdc-519725345b1e TargetServiceName=null EntryTimestamp=2022-04-26T06:23:46.797Z InvocationID=ebb75740-8071-412b-93cc-ee44c3166407 ErrorCode=500 Starting to create Csar for component contrailV2VLANSubInterfaceV2 2022-04-26T06:23:47.835Z [qtp2085002312-13] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "contrailV2VLANSubInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "ad83f439-9339-4405-a180-b96e227c9a4c" INVARIANT_UUID = "9ec3cc86-bab1-4dd9-be5f-98e4f2ef90db" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "13f8cc99-9ccf-4de6-ad3e-533f2b9886b9.assettoscatemplate" CURR_ARTIFACT_UUID = "13f8cc99-9ccf-4de6-ad3e-533f2b9886b9.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2vlansubinterfacev2-template.yml,0,13f8cc99-9ccf-4de6-ad3e-533f2b9886b9.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d9526003-3e5a-49c7-afdc-519725345b1e TargetServiceName=null EntryTimestamp=2022-04-26T06:23:46.797Z InvocationID=ebb75740-8071-412b-93cc-ee44c3166407 ErrorCode=500 Ended create Csar for component contrailV2VLANSubInterfaceV2 2022-04-26T06:23:47.835Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "contrailV2VLANSubInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "ad83f439-9339-4405-a180-b96e227c9a4c" INVARIANT_UUID = "9ec3cc86-bab1-4dd9-be5f-98e4f2ef90db" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "13f8cc99-9ccf-4de6-ad3e-533f2b9886b9.assettoscatemplate" CURR_ARTIFACT_UUID = "13f8cc99-9ccf-4de6-ad3e-533f2b9886b9.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2vlansubinterfacev2-template.yml,0,13f8cc99-9ccf-4de6-ad3e-533f2b9886b9.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d9526003-3e5a-49c7-afdc-519725345b1e TargetServiceName=null EntryTimestamp=2022-04-26T06:23:46.797Z InvocationID=ebb75740-8071-412b-93cc-ee44c3166407 ErrorCode=500 lock resource with id 13f8cc99-9ccf-4de6-ad3e-533f2b9886b9 2022-04-26T06:23:48.399Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "contrailV2VLANSubInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "ad83f439-9339-4405-a180-b96e227c9a4c" INVARIANT_UUID = "9ec3cc86-bab1-4dd9-be5f-98e4f2ef90db" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "13f8cc99-9ccf-4de6-ad3e-533f2b9886b9.assettoscacsar" CURR_ARTIFACT_UUID = "13f8cc99-9ccf-4de6-ad3e-533f2b9886b9.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Contrailv2vlansubinterfacev2-csar.csar,0,13f8cc99-9ccf-4de6-ad3e-533f2b9886b9.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1602 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9076b457-e8a1-49a7-a6be-dc35864afd7f TargetServiceName=null EntryTimestamp=2022-04-26T06:23:46.797Z InvocationID=ebb75740-8071-412b-93cc-ee44c3166407 ErrorCode=500 LogTimestamp=2022-04-26T06:23:48.399201Z ResponseDesc=Created Exiting. 2022-04-26T06:23:48.400Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:23:48.420Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=b0ec99b3-7c28-479d-94b7-9a1eacfaefcf InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:48.420450Z InvocationID=b7875938-d0ba-4071-91c7-c00d49555d74 LogTimestamp=2022-04-26T06:23:48.420633Z Entering 2022-04-26T06:23:48.421Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=b0ec99b3-7c28-479d-94b7-9a1eacfaefcf InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:48.420450Z InvocationID=b7875938-d0ba-4071-91c7-c00d49555d74 LogTimestamp=2022-04-26T06:23:48.420633Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:23:48.525Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:48.421Z InvocationID=259e9a65-1bc4-4d92-b06d-00671c3281ac PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:48.526Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:48.421Z InvocationID=259e9a65-1bc4-4d92-b06d-00671c3281ac PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:48.526Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:48.421Z InvocationID=259e9a65-1bc4-4d92-b06d-00671c3281ac PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:48.526Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:48.421Z InvocationID=259e9a65-1bc4-4d92-b06d-00671c3281ac PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:48.549Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:48.421Z InvocationID=259e9a65-1bc4-4d92-b06d-00671c3281ac PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name FabricConfiguration 2022-04-26T06:23:48.688Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Fabric Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "0c117392-651f-43f8-9d75-96ff5759038c" INVARIANT_UUID = "a1832eae-3db4-47d4-a613-693a72227350" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.nodes.FabricConfiguration" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0c117392-651f-43f8-9d75-96ff5759038c EntryTimestamp=2022-04-26T06:23:48.421Z InvocationID=259e9a65-1bc4-4d92-b06d-00671c3281ac PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 2ef90383-8bb0-4101-8ce6-32c04b2e82d7 2022-04-26T06:23:49.040Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e9d316c-ed58-4021-a7ec-b4ff93a945c0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:23:49.040Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e9d316c-ed58-4021-a7ec-b4ff93a945c0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3ddf9359 2022-04-26T06:23:49.041Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e9d316c-ed58-4021-a7ec-b4ff93a945c0 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:23:49.041Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e9d316c-ed58-4021-a7ec-b4ff93a945c0 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:23:49.652Z [qtp2085002312-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Fabric Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "0c117392-651f-43f8-9d75-96ff5759038c" INVARIANT_UUID = "a1832eae-3db4-47d4-a613-693a72227350" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2ef90383-8bb0-4101-8ce6-32c04b2e82d7.assettoscatemplate" CURR_ARTIFACT_UUID = "2ef90383-8bb0-4101-8ce6-32c04b2e82d7.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-FabricConfiguration-template.yml,0,2ef90383-8bb0-4101-8ce6-32c04b2e82d7.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6dd5e8ba-68ad-4ede-9676-b3bbfe6f4b62 EntryTimestamp=2022-04-26T06:23:48.421Z InvocationID=259e9a65-1bc4-4d92-b06d-00671c3281ac PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Starting to create Csar for component Fabric Configuration 2022-04-26T06:23:49.856Z [qtp2085002312-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Fabric Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "0c117392-651f-43f8-9d75-96ff5759038c" INVARIANT_UUID = "a1832eae-3db4-47d4-a613-693a72227350" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2ef90383-8bb0-4101-8ce6-32c04b2e82d7.assettoscatemplate" CURR_ARTIFACT_UUID = "2ef90383-8bb0-4101-8ce6-32c04b2e82d7.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-FabricConfiguration-template.yml,0,2ef90383-8bb0-4101-8ce6-32c04b2e82d7.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6dd5e8ba-68ad-4ede-9676-b3bbfe6f4b62 EntryTimestamp=2022-04-26T06:23:48.421Z InvocationID=259e9a65-1bc4-4d92-b06d-00671c3281ac Ended create Csar for component Fabric Configuration 2022-04-26T06:23:49.857Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Fabric Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "0c117392-651f-43f8-9d75-96ff5759038c" INVARIANT_UUID = "a1832eae-3db4-47d4-a613-693a72227350" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2ef90383-8bb0-4101-8ce6-32c04b2e82d7.assettoscatemplate" CURR_ARTIFACT_UUID = "2ef90383-8bb0-4101-8ce6-32c04b2e82d7.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-FabricConfiguration-template.yml,0,2ef90383-8bb0-4101-8ce6-32c04b2e82d7.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6dd5e8ba-68ad-4ede-9676-b3bbfe6f4b62 EntryTimestamp=2022-04-26T06:23:48.421Z InvocationID=259e9a65-1bc4-4d92-b06d-00671c3281ac lock resource with id 2ef90383-8bb0-4101-8ce6-32c04b2e82d7 2022-04-26T06:23:50.062Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=6e9d316c-ed58-4021-a7ec-b4ff93a945c0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:23:50.063Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=6e9d316c-ed58-4021-a7ec-b4ff93a945c0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:23:50.372Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:23:50.390Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:23:50.664Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Fabric Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "0c117392-651f-43f8-9d75-96ff5759038c" INVARIANT_UUID = "a1832eae-3db4-47d4-a613-693a72227350" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2ef90383-8bb0-4101-8ce6-32c04b2e82d7.assettoscacsar" CURR_ARTIFACT_UUID = "2ef90383-8bb0-4101-8ce6-32c04b2e82d7.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-FabricConfiguration-csar.csar,0,2ef90383-8bb0-4101-8ce6-32c04b2e82d7.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=2243 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=958c73ce-5618-4702-bc6f-dc2abbbf0a1d EntryTimestamp=2022-04-26T06:23:48.421Z InvocationID=259e9a65-1bc4-4d92-b06d-00671c3281ac LogTimestamp=2022-04-26T06:23:50.664095Z ResponseDesc=Created Exiting. 2022-04-26T06:23:50.665Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:23:50.744Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=70e9b3e5-81b5-4882-9a75-a6112ac8d701 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:50.744540Z InvocationID=8e76949a-3c82-4dbf-ac3e-831ef607317f LogTimestamp=2022-04-26T06:23:50.744670Z Entering 2022-04-26T06:23:50.745Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=70e9b3e5-81b5-4882-9a75-a6112ac8d701 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:50.744540Z InvocationID=8e76949a-3c82-4dbf-ac3e-831ef607317f LogTimestamp=2022-04-26T06:23:50.744670Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:23:51.244Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:50.746Z InvocationID=2fe18de9-70ee-44ac-b357-594adda1413b PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:51.244Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:50.746Z InvocationID=2fe18de9-70ee-44ac-b357-594adda1413b PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:51.244Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:50.746Z InvocationID=2fe18de9-70ee-44ac-b357-594adda1413b PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:51.244Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:50.746Z InvocationID=2fe18de9-70ee-44ac-b357-594adda1413b PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:51.251Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:50.746Z InvocationID=2fe18de9-70ee-44ac-b357-594adda1413b PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name VlUnderlayvpn 2022-04-26T06:23:51.545Z [qtp2085002312-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VL UNDERLAYVPN" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "3fb57bcd-9439-44d7-aa8c-f79207c7a12b" INVARIANT_UUID = "e1e89404-9b52-4c72-9b10-d0339a6e0019" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.underlayvpn" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3fb57bcd-9439-44d7-aa8c-f79207c7a12b EntryTimestamp=2022-04-26T06:23:50.746Z InvocationID=2fe18de9-70ee-44ac-b357-594adda1413b PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:23:51.546Z [qtp2085002312-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VL UNDERLAYVPN" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "3fb57bcd-9439-44d7-aa8c-f79207c7a12b" INVARIANT_UUID = "e1e89404-9b52-4c72-9b10-d0339a6e0019" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.underlayvpn" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3fb57bcd-9439-44d7-aa8c-f79207c7a12b TargetServiceName=null EntryTimestamp=2022-04-26T06:23:50.746Z InvocationID=2fe18de9-70ee-44ac-b357-594adda1413b ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:23:51.760Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VL UNDERLAYVPN" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "3fb57bcd-9439-44d7-aa8c-f79207c7a12b" INVARIANT_UUID = "e1e89404-9b52-4c72-9b10-d0339a6e0019" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.underlayvpn" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3fb57bcd-9439-44d7-aa8c-f79207c7a12b TargetServiceName=null EntryTimestamp=2022-04-26T06:23:50.746Z InvocationID=2fe18de9-70ee-44ac-b357-594adda1413b ErrorCode=500 lock resource with id 2bf8d051-cc69-4bb3-9b73-2e4bd4df597d 2022-04-26T06:23:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:23:52.686Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:23:52.686Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:23:52.842Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8cb0b88a-44f4-44d0-a985-f3cc374b233f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:23:52.843Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8cb0b88a-44f4-44d0-a985-f3cc374b233f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6b7d2d33 2022-04-26T06:23:52.843Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8cb0b88a-44f4-44d0-a985-f3cc374b233f PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:23:52.843Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8cb0b88a-44f4-44d0-a985-f3cc374b233f PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:23:53.440Z [qtp2085002312-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL UNDERLAYVPN" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "3fb57bcd-9439-44d7-aa8c-f79207c7a12b" INVARIANT_UUID = "e1e89404-9b52-4c72-9b10-d0339a6e0019" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2bf8d051-cc69-4bb3-9b73-2e4bd4df597d.assettoscatemplate" CURR_ARTIFACT_UUID = "2bf8d051-cc69-4bb3-9b73-2e4bd4df597d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VlUnderlayvpn-template.yml,0,2bf8d051-cc69-4bb3-9b73-2e4bd4df597d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7760ac0d-acc4-4648-9240-db968cbee6fa TargetServiceName=null EntryTimestamp=2022-04-26T06:23:50.746Z InvocationID=2fe18de9-70ee-44ac-b357-594adda1413b ErrorCode=500 Starting to create Csar for component VL UNDERLAYVPN 2022-04-26T06:23:53.635Z [qtp2085002312-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL UNDERLAYVPN" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "3fb57bcd-9439-44d7-aa8c-f79207c7a12b" INVARIANT_UUID = "e1e89404-9b52-4c72-9b10-d0339a6e0019" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2bf8d051-cc69-4bb3-9b73-2e4bd4df597d.assettoscatemplate" CURR_ARTIFACT_UUID = "2bf8d051-cc69-4bb3-9b73-2e4bd4df597d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VlUnderlayvpn-template.yml,0,2bf8d051-cc69-4bb3-9b73-2e4bd4df597d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7760ac0d-acc4-4648-9240-db968cbee6fa TargetServiceName=null EntryTimestamp=2022-04-26T06:23:50.746Z InvocationID=2fe18de9-70ee-44ac-b357-594adda1413b ErrorCode=500 Ended create Csar for component VL UNDERLAYVPN 2022-04-26T06:23:53.635Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL UNDERLAYVPN" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "3fb57bcd-9439-44d7-aa8c-f79207c7a12b" INVARIANT_UUID = "e1e89404-9b52-4c72-9b10-d0339a6e0019" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2bf8d051-cc69-4bb3-9b73-2e4bd4df597d.assettoscatemplate" CURR_ARTIFACT_UUID = "2bf8d051-cc69-4bb3-9b73-2e4bd4df597d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VlUnderlayvpn-template.yml,0,2bf8d051-cc69-4bb3-9b73-2e4bd4df597d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7760ac0d-acc4-4648-9240-db968cbee6fa TargetServiceName=null EntryTimestamp=2022-04-26T06:23:50.746Z InvocationID=2fe18de9-70ee-44ac-b357-594adda1413b ErrorCode=500 lock resource with id 2bf8d051-cc69-4bb3-9b73-2e4bd4df597d 2022-04-26T06:23:53.865Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=8cb0b88a-44f4-44d0-a985-f3cc374b233f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:23:53.866Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=8cb0b88a-44f4-44d0-a985-f3cc374b233f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:23:54.356Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL UNDERLAYVPN" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "3fb57bcd-9439-44d7-aa8c-f79207c7a12b" INVARIANT_UUID = "e1e89404-9b52-4c72-9b10-d0339a6e0019" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2bf8d051-cc69-4bb3-9b73-2e4bd4df597d.assettoscacsar" CURR_ARTIFACT_UUID = "2bf8d051-cc69-4bb3-9b73-2e4bd4df597d.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VlUnderlayvpn-csar.csar,0,2bf8d051-cc69-4bb3-9b73-2e4bd4df597d.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=3610 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=cbb9bb4e-1fea-4619-909a-b209a0e1263a TargetServiceName=null EntryTimestamp=2022-04-26T06:23:50.746Z InvocationID=2fe18de9-70ee-44ac-b357-594adda1413b ErrorCode=500 LogTimestamp=2022-04-26T06:23:54.356337Z ResponseDesc=Created Exiting. 2022-04-26T06:23:54.357Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:23:54.444Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c64f66bd-6a89-4b57-a57a-153948fc4890 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:54.444090Z InvocationID=9df52a15-8e4f-49e7-bfb2-b3b5cc604d34 LogTimestamp=2022-04-26T06:23:54.444198Z Entering 2022-04-26T06:23:54.444Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c64f66bd-6a89-4b57-a57a-153948fc4890 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:54.444090Z InvocationID=9df52a15-8e4f-49e7-bfb2-b3b5cc604d34 LogTimestamp=2022-04-26T06:23:54.444198Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:23:55.346Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:54.445Z InvocationID=83ed5bc4-c70a-468a-8689-ef53f6241cce PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:55.347Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:54.445Z InvocationID=83ed5bc4-c70a-468a-8689-ef53f6241cce PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:55.347Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:54.445Z InvocationID=83ed5bc4-c70a-468a-8689-ef53f6241cce PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:55.347Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:54.445Z InvocationID=83ed5bc4-c70a-468a-8689-ef53f6241cce PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:23:55.448Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:54.445Z InvocationID=83ed5bc4-c70a-468a-8689-ef53f6241cce PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name VlOverlaytunnel 2022-04-26T06:23:55.549Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd5c8784-367f-4465-a971-2feecc83f954 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:23:55.549Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd5c8784-367f-4465-a971-2feecc83f954 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@214d117e 2022-04-26T06:23:55.550Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd5c8784-367f-4465-a971-2feecc83f954 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:23:55.550Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd5c8784-367f-4465-a971-2feecc83f954 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:23:55.941Z [qtp2085002312-20] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VL OVERLAYTUNNEL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "ebb2205e-78d2-456c-a765-0335fe5c4de2" INVARIANT_UUID = "5598273e-37c1-4d3a-944c-7eabd8b88cd0" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.overlaytunnel" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ebb2205e-78d2-456c-a765-0335fe5c4de2 EntryTimestamp=2022-04-26T06:23:54.445Z InvocationID=83ed5bc4-c70a-468a-8689-ef53f6241cce PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:23:55.942Z [qtp2085002312-20] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VL OVERLAYTUNNEL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "ebb2205e-78d2-456c-a765-0335fe5c4de2" INVARIANT_UUID = "5598273e-37c1-4d3a-944c-7eabd8b88cd0" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.overlaytunnel" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ebb2205e-78d2-456c-a765-0335fe5c4de2 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:54.445Z InvocationID=83ed5bc4-c70a-468a-8689-ef53f6241cce ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:23:56.159Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VL OVERLAYTUNNEL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "ebb2205e-78d2-456c-a765-0335fe5c4de2" INVARIANT_UUID = "5598273e-37c1-4d3a-944c-7eabd8b88cd0" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.overlaytunnel" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ebb2205e-78d2-456c-a765-0335fe5c4de2 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:54.445Z InvocationID=83ed5bc4-c70a-468a-8689-ef53f6241cce ErrorCode=500 lock resource with id 8da665e4-0242-4a3b-ac85-3f3dc1dc9f7d 2022-04-26T06:23:56.595Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=fd5c8784-367f-4465-a971-2feecc83f954 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:23:56.596Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=fd5c8784-367f-4465-a971-2feecc83f954 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:23:57.449Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL OVERLAYTUNNEL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "ebb2205e-78d2-456c-a765-0335fe5c4de2" INVARIANT_UUID = "5598273e-37c1-4d3a-944c-7eabd8b88cd0" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8da665e4-0242-4a3b-ac85-3f3dc1dc9f7d.assettoscatemplate" CURR_ARTIFACT_UUID = "8da665e4-0242-4a3b-ac85-3f3dc1dc9f7d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VlOverlaytunnel-template.yml,0,8da665e4-0242-4a3b-ac85-3f3dc1dc9f7d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b52cfa6b-28fa-4f37-9b8b-c9f7ce49ba35 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:54.445Z InvocationID=83ed5bc4-c70a-468a-8689-ef53f6241cce ErrorCode=500 Starting to create Csar for component VL OVERLAYTUNNEL 2022-04-26T06:23:57.784Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL OVERLAYTUNNEL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "ebb2205e-78d2-456c-a765-0335fe5c4de2" INVARIANT_UUID = "5598273e-37c1-4d3a-944c-7eabd8b88cd0" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8da665e4-0242-4a3b-ac85-3f3dc1dc9f7d.assettoscatemplate" CURR_ARTIFACT_UUID = "8da665e4-0242-4a3b-ac85-3f3dc1dc9f7d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VlOverlaytunnel-template.yml,0,8da665e4-0242-4a3b-ac85-3f3dc1dc9f7d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b52cfa6b-28fa-4f37-9b8b-c9f7ce49ba35 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:54.445Z InvocationID=83ed5bc4-c70a-468a-8689-ef53f6241cce ErrorCode=500 Ended create Csar for component VL OVERLAYTUNNEL 2022-04-26T06:23:57.785Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL OVERLAYTUNNEL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "ebb2205e-78d2-456c-a765-0335fe5c4de2" INVARIANT_UUID = "5598273e-37c1-4d3a-944c-7eabd8b88cd0" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8da665e4-0242-4a3b-ac85-3f3dc1dc9f7d.assettoscatemplate" CURR_ARTIFACT_UUID = "8da665e4-0242-4a3b-ac85-3f3dc1dc9f7d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VlOverlaytunnel-template.yml,0,8da665e4-0242-4a3b-ac85-3f3dc1dc9f7d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b52cfa6b-28fa-4f37-9b8b-c9f7ce49ba35 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:54.445Z InvocationID=83ed5bc4-c70a-468a-8689-ef53f6241cce ErrorCode=500 lock resource with id 8da665e4-0242-4a3b-ac85-3f3dc1dc9f7d 2022-04-26T06:23:58.347Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c45b220e-297a-4c1a-be45-838c1c2eae75 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:23:58.347Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c45b220e-297a-4c1a-be45-838c1c2eae75 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1da40957 2022-04-26T06:23:58.347Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c45b220e-297a-4c1a-be45-838c1c2eae75 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:23:58.347Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c45b220e-297a-4c1a-be45-838c1c2eae75 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:23:58.557Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL OVERLAYTUNNEL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "ebb2205e-78d2-456c-a765-0335fe5c4de2" INVARIANT_UUID = "5598273e-37c1-4d3a-944c-7eabd8b88cd0" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8da665e4-0242-4a3b-ac85-3f3dc1dc9f7d.assettoscacsar" CURR_ARTIFACT_UUID = "8da665e4-0242-4a3b-ac85-3f3dc1dc9f7d.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VlOverlaytunnel-csar.csar,0,8da665e4-0242-4a3b-ac85-3f3dc1dc9f7d.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=4111 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0f9dec22-6f7c-4edb-a42f-8fee1ccf2a2e TargetServiceName=null EntryTimestamp=2022-04-26T06:23:54.445Z InvocationID=83ed5bc4-c70a-468a-8689-ef53f6241cce ErrorCode=500 LogTimestamp=2022-04-26T06:23:58.556872Z ResponseDesc=Created Exiting. 2022-04-26T06:23:58.557Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:23:58.581Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a47731b6-495e-4b81-bc26-a3c9fe09fec8 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:58.581039Z InvocationID=41405467-ef22-4e28-bf55-b6fb263e7f70 LogTimestamp=2022-04-26T06:23:58.581131Z Entering 2022-04-26T06:23:58.581Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a47731b6-495e-4b81-bc26-a3c9fe09fec8 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:23:58.581039Z InvocationID=41405467-ef22-4e28-bf55-b6fb263e7f70 LogTimestamp=2022-04-26T06:23:58.581131Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:23:59.374Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c45b220e-297a-4c1a-be45-838c1c2eae75 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:23:59.375Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c45b220e-297a-4c1a-be45-838c1c2eae75 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:24:01.456Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:58.582Z InvocationID=dcbec728-c24e-42b1-bcb5-461f4d351259 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:01.457Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:58.582Z InvocationID=dcbec728-c24e-42b1-bcb5-461f4d351259 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:01.457Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:58.582Z InvocationID=dcbec728-c24e-42b1-bcb5-461f4d351259 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:01.457Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:58.582Z InvocationID=dcbec728-c24e-42b1-bcb5-461f4d351259 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:01.542Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:23:58.582Z InvocationID=dcbec728-c24e-42b1-bcb5-461f4d351259 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name GenericNeutronnet 2022-04-26T06:24:01.543Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3344e459-1188-4d35-b234-c403756a3e24 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:24:01.544Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3344e459-1188-4d35-b234-c403756a3e24 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7c51d219 2022-04-26T06:24:01.544Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3344e459-1188-4d35-b234-c403756a3e24 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:24:01.544Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3344e459-1188-4d35-b234-c403756a3e24 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:24:02.242Z [qtp2085002312-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Generic NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "2507fecc-1d16-49d0-a12e-d4a484630d8e" INVARIANT_UUID = "e7cd06ca-1bb2-47e0-a38f-e59ca0aabc1b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.GenericNeutronNet" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2507fecc-1d16-49d0-a12e-d4a484630d8e EntryTimestamp=2022-04-26T06:23:58.582Z InvocationID=dcbec728-c24e-42b1-bcb5-461f4d351259 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:24:02.243Z [qtp2085002312-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Generic NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "2507fecc-1d16-49d0-a12e-d4a484630d8e" INVARIANT_UUID = "e7cd06ca-1bb2-47e0-a38f-e59ca0aabc1b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.GenericNeutronNet" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2507fecc-1d16-49d0-a12e-d4a484630d8e TargetServiceName=null EntryTimestamp=2022-04-26T06:23:58.582Z InvocationID=dcbec728-c24e-42b1-bcb5-461f4d351259 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:24:02.573Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3344e459-1188-4d35-b234-c403756a3e24 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:24:02.574Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3344e459-1188-4d35-b234-c403756a3e24 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:24:02.757Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Generic NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "2507fecc-1d16-49d0-a12e-d4a484630d8e" INVARIANT_UUID = "e7cd06ca-1bb2-47e0-a38f-e59ca0aabc1b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.GenericNeutronNet" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2507fecc-1d16-49d0-a12e-d4a484630d8e TargetServiceName=null EntryTimestamp=2022-04-26T06:23:58.582Z InvocationID=dcbec728-c24e-42b1-bcb5-461f4d351259 ErrorCode=500 lock resource with id c820d871-8f27-4972-b942-0db2c881cecd 2022-04-26T06:24:04.342Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4dbb16ed-c703-4b38-992d-acc34a5b687b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:24:04.342Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4dbb16ed-c703-4b38-992d-acc34a5b687b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4b636497 2022-04-26T06:24:04.343Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4dbb16ed-c703-4b38-992d-acc34a5b687b PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:24:04.343Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4dbb16ed-c703-4b38-992d-acc34a5b687b PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:24:05.034Z [qtp2085002312-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Generic NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "2507fecc-1d16-49d0-a12e-d4a484630d8e" INVARIANT_UUID = "e7cd06ca-1bb2-47e0-a38f-e59ca0aabc1b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c820d871-8f27-4972-b942-0db2c881cecd.assettoscatemplate" CURR_ARTIFACT_UUID = "c820d871-8f27-4972-b942-0db2c881cecd.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-GenericNeutronnet-template.yml,0,c820d871-8f27-4972-b942-0db2c881cecd.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=92a9694a-3f24-4e80-9b9e-7a2e4cb7d515 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:58.582Z InvocationID=dcbec728-c24e-42b1-bcb5-461f4d351259 ErrorCode=500 Starting to create Csar for component Generic NeutronNet 2022-04-26T06:24:05.303Z [qtp2085002312-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Generic NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "2507fecc-1d16-49d0-a12e-d4a484630d8e" INVARIANT_UUID = "e7cd06ca-1bb2-47e0-a38f-e59ca0aabc1b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c820d871-8f27-4972-b942-0db2c881cecd.assettoscatemplate" CURR_ARTIFACT_UUID = "c820d871-8f27-4972-b942-0db2c881cecd.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-GenericNeutronnet-template.yml,0,c820d871-8f27-4972-b942-0db2c881cecd.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=92a9694a-3f24-4e80-9b9e-7a2e4cb7d515 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:58.582Z InvocationID=dcbec728-c24e-42b1-bcb5-461f4d351259 ErrorCode=500 Ended create Csar for component Generic NeutronNet 2022-04-26T06:24:05.304Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Generic NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "2507fecc-1d16-49d0-a12e-d4a484630d8e" INVARIANT_UUID = "e7cd06ca-1bb2-47e0-a38f-e59ca0aabc1b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c820d871-8f27-4972-b942-0db2c881cecd.assettoscatemplate" CURR_ARTIFACT_UUID = "c820d871-8f27-4972-b942-0db2c881cecd.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-GenericNeutronnet-template.yml,0,c820d871-8f27-4972-b942-0db2c881cecd.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=92a9694a-3f24-4e80-9b9e-7a2e4cb7d515 TargetServiceName=null EntryTimestamp=2022-04-26T06:23:58.582Z InvocationID=dcbec728-c24e-42b1-bcb5-461f4d351259 ErrorCode=500 lock resource with id c820d871-8f27-4972-b942-0db2c881cecd 2022-04-26T06:24:05.353Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=4dbb16ed-c703-4b38-992d-acc34a5b687b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:24:05.355Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=4dbb16ed-c703-4b38-992d-acc34a5b687b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:24:05.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:24:05.384Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:24:06.453Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Generic NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "2507fecc-1d16-49d0-a12e-d4a484630d8e" INVARIANT_UUID = "e7cd06ca-1bb2-47e0-a38f-e59ca0aabc1b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c820d871-8f27-4972-b942-0db2c881cecd.assettoscacsar" CURR_ARTIFACT_UUID = "c820d871-8f27-4972-b942-0db2c881cecd.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-GenericNeutronnet-csar.csar,0,c820d871-8f27-4972-b942-0db2c881cecd.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=7870 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bdb8eaa5-1aa9-413a-9721-f412a3d0cc5d TargetServiceName=null EntryTimestamp=2022-04-26T06:23:58.582Z InvocationID=dcbec728-c24e-42b1-bcb5-461f4d351259 ErrorCode=500 LogTimestamp=2022-04-26T06:24:06.452900Z ResponseDesc=Created Exiting. 2022-04-26T06:24:06.453Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:24:06.468Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=03feeb0b-4e9d-4b27-9aa0-9787303e98c8 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:06.468633Z InvocationID=59744ee3-6982-4ff6-8e44-cadc78b0d565 LogTimestamp=2022-04-26T06:24:06.468712Z Entering 2022-04-26T06:24:06.469Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=03feeb0b-4e9d-4b27-9aa0-9787303e98c8 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:06.468633Z InvocationID=59744ee3-6982-4ff6-8e44-cadc78b0d565 LogTimestamp=2022-04-26T06:24:06.468712Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:24:07.552Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:06.469Z InvocationID=9ca1467b-de92-45ee-a956-e445f826d64a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:07.552Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:06.469Z InvocationID=9ca1467b-de92-45ee-a956-e445f826d64a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:07.552Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:06.469Z InvocationID=9ca1467b-de92-45ee-a956-e445f826d64a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:07.552Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:06.469Z InvocationID=9ca1467b-de92-45ee-a956-e445f826d64a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:07.685Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:24:07.737Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:06.469Z InvocationID=9ca1467b-de92-45ee-a956-e445f826d64a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Allottedresource 2022-04-26T06:24:07.833Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=13589e10-943d-4cdd-937a-6a4a320e05c4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:24:07.833Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=13589e10-943d-4cdd-937a-6a4a320e05c4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7bd5fbe2 2022-04-26T06:24:07.833Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=13589e10-943d-4cdd-937a-6a4a320e05c4 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:24:07.834Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=13589e10-943d-4cdd-937a-6a4a320e05c4 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:24:08.230Z [qtp2085002312-20] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "AllottedResource" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "6301686f-87ee-4370-8cba-a82157325b5c" INVARIANT_UUID = "385287db-b2a4-4e30-a190-1851652a11c7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.AllottedResource" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6301686f-87ee-4370-8cba-a82157325b5c EntryTimestamp=2022-04-26T06:24:06.469Z InvocationID=9ca1467b-de92-45ee-a956-e445f826d64a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:24:08.232Z [qtp2085002312-20] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "AllottedResource" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "6301686f-87ee-4370-8cba-a82157325b5c" INVARIANT_UUID = "385287db-b2a4-4e30-a190-1851652a11c7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.AllottedResource" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6301686f-87ee-4370-8cba-a82157325b5c TargetServiceName=null EntryTimestamp=2022-04-26T06:24:06.469Z InvocationID=9ca1467b-de92-45ee-a956-e445f826d64a ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:24:08.550Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "AllottedResource" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "6301686f-87ee-4370-8cba-a82157325b5c" INVARIANT_UUID = "385287db-b2a4-4e30-a190-1851652a11c7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.AllottedResource" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6301686f-87ee-4370-8cba-a82157325b5c TargetServiceName=null EntryTimestamp=2022-04-26T06:24:06.469Z InvocationID=9ca1467b-de92-45ee-a956-e445f826d64a ErrorCode=500 lock resource with id ce7c29b5-c4f8-465c-acb4-cce12db10a93 2022-04-26T06:24:08.877Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=13589e10-943d-4cdd-937a-6a4a320e05c4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:24:08.878Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=13589e10-943d-4cdd-937a-6a4a320e05c4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:24:10.234Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "AllottedResource" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "6301686f-87ee-4370-8cba-a82157325b5c" INVARIANT_UUID = "385287db-b2a4-4e30-a190-1851652a11c7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ce7c29b5-c4f8-465c-acb4-cce12db10a93.assettoscatemplate" CURR_ARTIFACT_UUID = "ce7c29b5-c4f8-465c-acb4-cce12db10a93.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Allottedresource-template.yml,0,ce7c29b5-c4f8-465c-acb4-cce12db10a93.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b6c6268a-f4e4-45b0-aa4f-5e646f512f9e TargetServiceName=null EntryTimestamp=2022-04-26T06:24:06.469Z InvocationID=9ca1467b-de92-45ee-a956-e445f826d64a ErrorCode=500 Starting to create Csar for component AllottedResource 2022-04-26T06:24:10.272Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8def105-4c5d-4bb1-99dc-660238eec540 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:24:10.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8def105-4c5d-4bb1-99dc-660238eec540 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5c286efe 2022-04-26T06:24:10.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8def105-4c5d-4bb1-99dc-660238eec540 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:24:10.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8def105-4c5d-4bb1-99dc-660238eec540 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:24:10.456Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "AllottedResource" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "6301686f-87ee-4370-8cba-a82157325b5c" INVARIANT_UUID = "385287db-b2a4-4e30-a190-1851652a11c7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ce7c29b5-c4f8-465c-acb4-cce12db10a93.assettoscatemplate" CURR_ARTIFACT_UUID = "ce7c29b5-c4f8-465c-acb4-cce12db10a93.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Allottedresource-template.yml,0,ce7c29b5-c4f8-465c-acb4-cce12db10a93.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b6c6268a-f4e4-45b0-aa4f-5e646f512f9e TargetServiceName=null EntryTimestamp=2022-04-26T06:24:06.469Z InvocationID=9ca1467b-de92-45ee-a956-e445f826d64a ErrorCode=500 Ended create Csar for component AllottedResource 2022-04-26T06:24:10.456Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "AllottedResource" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "6301686f-87ee-4370-8cba-a82157325b5c" INVARIANT_UUID = "385287db-b2a4-4e30-a190-1851652a11c7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ce7c29b5-c4f8-465c-acb4-cce12db10a93.assettoscatemplate" CURR_ARTIFACT_UUID = "ce7c29b5-c4f8-465c-acb4-cce12db10a93.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Allottedresource-template.yml,0,ce7c29b5-c4f8-465c-acb4-cce12db10a93.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b6c6268a-f4e4-45b0-aa4f-5e646f512f9e TargetServiceName=null EntryTimestamp=2022-04-26T06:24:06.469Z InvocationID=9ca1467b-de92-45ee-a956-e445f826d64a ErrorCode=500 lock resource with id ce7c29b5-c4f8-465c-acb4-cce12db10a93 2022-04-26T06:24:11.012Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "AllottedResource" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "6301686f-87ee-4370-8cba-a82157325b5c" INVARIANT_UUID = "385287db-b2a4-4e30-a190-1851652a11c7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ce7c29b5-c4f8-465c-acb4-cce12db10a93.assettoscacsar" CURR_ARTIFACT_UUID = "ce7c29b5-c4f8-465c-acb4-cce12db10a93.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Allottedresource-csar.csar,0,ce7c29b5-c4f8-465c-acb4-cce12db10a93.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=4543 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9d15ba7f-4058-45d4-9271-244ccc2d04df TargetServiceName=null EntryTimestamp=2022-04-26T06:24:06.469Z InvocationID=9ca1467b-de92-45ee-a956-e445f826d64a ErrorCode=500 LogTimestamp=2022-04-26T06:24:11.012351Z ResponseDesc=Created Exiting. 2022-04-26T06:24:11.013Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:24:11.024Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=22f2dff6-684d-4fd5-af95-043831be8223 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:11.024671Z InvocationID=c3375592-c140-4ac0-8d2e-f7c4a823fc68 LogTimestamp=2022-04-26T06:24:11.024741Z Entering 2022-04-26T06:24:11.025Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=22f2dff6-684d-4fd5-af95-043831be8223 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:11.024671Z InvocationID=c3375592-c140-4ac0-8d2e-f7c4a823fc68 LogTimestamp=2022-04-26T06:24:11.024741Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:24:11.186Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:11.025Z InvocationID=93aa5647-1d7c-4d7e-b881-820ad83f1e4c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:11.187Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:11.025Z InvocationID=93aa5647-1d7c-4d7e-b881-820ad83f1e4c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:11.187Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:11.025Z InvocationID=93aa5647-1d7c-4d7e-b881-820ad83f1e4c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:11.187Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:11.025Z InvocationID=93aa5647-1d7c-4d7e-b881-820ad83f1e4c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:11.207Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:11.025Z InvocationID=93aa5647-1d7c-4d7e-b881-820ad83f1e4c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name ExtImageFile 2022-04-26T06:24:11.245Z [qtp2085002312-13] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Ext Image File" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "0e2389ce-52a3-4dc7-9596-d911042ef022" INVARIANT_UUID = "b6942ce7-c30b-4538-8f25-6c9cf7029af4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.ImageFile" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0e2389ce-52a3-4dc7-9596-d911042ef022 EntryTimestamp=2022-04-26T06:24:11.025Z InvocationID=93aa5647-1d7c-4d7e-b881-820ad83f1e4c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:24:11.246Z [qtp2085002312-13] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Ext Image File" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "0e2389ce-52a3-4dc7-9596-d911042ef022" INVARIANT_UUID = "b6942ce7-c30b-4538-8f25-6c9cf7029af4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.ImageFile" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0e2389ce-52a3-4dc7-9596-d911042ef022 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:11.025Z InvocationID=93aa5647-1d7c-4d7e-b881-820ad83f1e4c ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:24:11.305Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f8def105-4c5d-4bb1-99dc-660238eec540 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:24:11.305Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f8def105-4c5d-4bb1-99dc-660238eec540 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:24:11.457Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Ext Image File" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "0e2389ce-52a3-4dc7-9596-d911042ef022" INVARIANT_UUID = "b6942ce7-c30b-4538-8f25-6c9cf7029af4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.ImageFile" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0e2389ce-52a3-4dc7-9596-d911042ef022 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:11.025Z InvocationID=93aa5647-1d7c-4d7e-b881-820ad83f1e4c ErrorCode=500 lock resource with id f9c4be7c-6b0d-4fa1-a1ce-acb4f3de504b 2022-04-26T06:24:12.140Z [qtp2085002312-13] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Ext Image File" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "0e2389ce-52a3-4dc7-9596-d911042ef022" INVARIANT_UUID = "b6942ce7-c30b-4538-8f25-6c9cf7029af4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "f9c4be7c-6b0d-4fa1-a1ce-acb4f3de504b.assettoscatemplate" CURR_ARTIFACT_UUID = "f9c4be7c-6b0d-4fa1-a1ce-acb4f3de504b.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtImageFile-template.yml,0,f9c4be7c-6b0d-4fa1-a1ce-acb4f3de504b.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b45d7daa-327a-45ad-b678-52efc44ff4f5 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:11.025Z InvocationID=93aa5647-1d7c-4d7e-b881-820ad83f1e4c ErrorCode=500 Starting to create Csar for component Ext Image File 2022-04-26T06:24:12.367Z [qtp2085002312-13] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Ext Image File" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "0e2389ce-52a3-4dc7-9596-d911042ef022" INVARIANT_UUID = "b6942ce7-c30b-4538-8f25-6c9cf7029af4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "f9c4be7c-6b0d-4fa1-a1ce-acb4f3de504b.assettoscatemplate" CURR_ARTIFACT_UUID = "f9c4be7c-6b0d-4fa1-a1ce-acb4f3de504b.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtImageFile-template.yml,0,f9c4be7c-6b0d-4fa1-a1ce-acb4f3de504b.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b45d7daa-327a-45ad-b678-52efc44ff4f5 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:11.025Z InvocationID=93aa5647-1d7c-4d7e-b881-820ad83f1e4c ErrorCode=500 Ended create Csar for component Ext Image File 2022-04-26T06:24:12.368Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Ext Image File" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "0e2389ce-52a3-4dc7-9596-d911042ef022" INVARIANT_UUID = "b6942ce7-c30b-4538-8f25-6c9cf7029af4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "f9c4be7c-6b0d-4fa1-a1ce-acb4f3de504b.assettoscatemplate" CURR_ARTIFACT_UUID = "f9c4be7c-6b0d-4fa1-a1ce-acb4f3de504b.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtImageFile-template.yml,0,f9c4be7c-6b0d-4fa1-a1ce-acb4f3de504b.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b45d7daa-327a-45ad-b678-52efc44ff4f5 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:11.025Z InvocationID=93aa5647-1d7c-4d7e-b881-820ad83f1e4c ErrorCode=500 lock resource with id f9c4be7c-6b0d-4fa1-a1ce-acb4f3de504b 2022-04-26T06:24:12.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6279ed0-ecb7-48aa-abc7-13d53172d93e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:24:12.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6279ed0-ecb7-48aa-abc7-13d53172d93e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@302629e9 2022-04-26T06:24:12.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6279ed0-ecb7-48aa-abc7-13d53172d93e PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:24:12.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6279ed0-ecb7-48aa-abc7-13d53172d93e PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:24:12.923Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Ext Image File" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "0e2389ce-52a3-4dc7-9596-d911042ef022" INVARIANT_UUID = "b6942ce7-c30b-4538-8f25-6c9cf7029af4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "f9c4be7c-6b0d-4fa1-a1ce-acb4f3de504b.assettoscacsar" CURR_ARTIFACT_UUID = "f9c4be7c-6b0d-4fa1-a1ce-acb4f3de504b.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-ExtImageFile-csar.csar,0,f9c4be7c-6b0d-4fa1-a1ce-acb4f3de504b.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1898 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=dd479758-04c9-4cea-ad1a-0a88dfcc2627 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:11.025Z InvocationID=93aa5647-1d7c-4d7e-b881-820ad83f1e4c ErrorCode=500 LogTimestamp=2022-04-26T06:24:12.923519Z ResponseDesc=Created Exiting. 2022-04-26T06:24:12.924Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:24:12.936Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=74e2c5a7-9f48-4975-90a7-28c75a1d2a6d InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:12.936470Z InvocationID=ad7a9e4e-b63d-46f7-82f9-7a9ecfe9ee6b LogTimestamp=2022-04-26T06:24:12.936539Z Entering 2022-04-26T06:24:12.937Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=74e2c5a7-9f48-4975-90a7-28c75a1d2a6d InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:12.936470Z InvocationID=ad7a9e4e-b63d-46f7-82f9-7a9ecfe9ee6b LogTimestamp=2022-04-26T06:24:12.936539Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:24:13.041Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:12.937Z InvocationID=b901a338-cba0-4bd1-be64-0e718ac6c6a7 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:13.042Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:12.937Z InvocationID=b901a338-cba0-4bd1-be64-0e718ac6c6a7 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:13.042Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:12.937Z InvocationID=b901a338-cba0-4bd1-be64-0e718ac6c6a7 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:13.042Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:12.937Z InvocationID=b901a338-cba0-4bd1-be64-0e718ac6c6a7 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:13.050Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:12.937Z InvocationID=b901a338-cba0-4bd1-be64-0e718ac6c6a7 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name ExtLocalStorage 2022-04-26T06:24:13.098Z [qtp2085002312-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Ext Local Storage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "430b5e81-1df4-4f92-974a-0aa6d99f406a" INVARIANT_UUID = "7a9fe088-f01b-4594-90a4-1929c09a85b9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.LocalStorage" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=430b5e81-1df4-4f92-974a-0aa6d99f406a EntryTimestamp=2022-04-26T06:24:12.937Z InvocationID=b901a338-cba0-4bd1-be64-0e718ac6c6a7 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:24:13.100Z [qtp2085002312-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Ext Local Storage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "430b5e81-1df4-4f92-974a-0aa6d99f406a" INVARIANT_UUID = "7a9fe088-f01b-4594-90a4-1929c09a85b9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.LocalStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=430b5e81-1df4-4f92-974a-0aa6d99f406a TargetServiceName=null EntryTimestamp=2022-04-26T06:24:12.937Z InvocationID=b901a338-cba0-4bd1-be64-0e718ac6c6a7 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:24:13.157Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Ext Local Storage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "430b5e81-1df4-4f92-974a-0aa6d99f406a" INVARIANT_UUID = "7a9fe088-f01b-4594-90a4-1929c09a85b9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.LocalStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=430b5e81-1df4-4f92-974a-0aa6d99f406a TargetServiceName=null EntryTimestamp=2022-04-26T06:24:12.937Z InvocationID=b901a338-cba0-4bd1-be64-0e718ac6c6a7 ErrorCode=500 lock resource with id c214f45f-07ed-4423-b274-90f6ac789549 2022-04-26T06:24:13.777Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Ext Local Storage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "430b5e81-1df4-4f92-974a-0aa6d99f406a" INVARIANT_UUID = "7a9fe088-f01b-4594-90a4-1929c09a85b9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c214f45f-07ed-4423-b274-90f6ac789549.assettoscatemplate" CURR_ARTIFACT_UUID = "c214f45f-07ed-4423-b274-90f6ac789549.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtLocalStorage-template.yml,0,c214f45f-07ed-4423-b274-90f6ac789549.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6df01508-0209-472a-b4b5-f69fa21f5842 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:12.937Z InvocationID=b901a338-cba0-4bd1-be64-0e718ac6c6a7 ErrorCode=500 Starting to create Csar for component Ext Local Storage 2022-04-26T06:24:13.896Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c6279ed0-ecb7-48aa-abc7-13d53172d93e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:24:13.897Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c6279ed0-ecb7-48aa-abc7-13d53172d93e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:24:13.931Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Ext Local Storage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "430b5e81-1df4-4f92-974a-0aa6d99f406a" INVARIANT_UUID = "7a9fe088-f01b-4594-90a4-1929c09a85b9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c214f45f-07ed-4423-b274-90f6ac789549.assettoscatemplate" CURR_ARTIFACT_UUID = "c214f45f-07ed-4423-b274-90f6ac789549.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtLocalStorage-template.yml,0,c214f45f-07ed-4423-b274-90f6ac789549.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6df01508-0209-472a-b4b5-f69fa21f5842 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:12.937Z InvocationID=b901a338-cba0-4bd1-be64-0e718ac6c6a7 ErrorCode=500 Ended create Csar for component Ext Local Storage 2022-04-26T06:24:13.932Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Ext Local Storage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "430b5e81-1df4-4f92-974a-0aa6d99f406a" INVARIANT_UUID = "7a9fe088-f01b-4594-90a4-1929c09a85b9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c214f45f-07ed-4423-b274-90f6ac789549.assettoscatemplate" CURR_ARTIFACT_UUID = "c214f45f-07ed-4423-b274-90f6ac789549.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtLocalStorage-template.yml,0,c214f45f-07ed-4423-b274-90f6ac789549.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6df01508-0209-472a-b4b5-f69fa21f5842 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:12.937Z InvocationID=b901a338-cba0-4bd1-be64-0e718ac6c6a7 ErrorCode=500 lock resource with id c214f45f-07ed-4423-b274-90f6ac789549 2022-04-26T06:24:14.488Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Ext Local Storage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "430b5e81-1df4-4f92-974a-0aa6d99f406a" INVARIANT_UUID = "7a9fe088-f01b-4594-90a4-1929c09a85b9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c214f45f-07ed-4423-b274-90f6ac789549.assettoscacsar" CURR_ARTIFACT_UUID = "c214f45f-07ed-4423-b274-90f6ac789549.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-ExtLocalStorage-csar.csar,0,c214f45f-07ed-4423-b274-90f6ac789549.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1551 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=204389e2-4899-4c97-a672-fadfaa12096f TargetServiceName=null EntryTimestamp=2022-04-26T06:24:12.937Z InvocationID=b901a338-cba0-4bd1-be64-0e718ac6c6a7 ErrorCode=500 LogTimestamp=2022-04-26T06:24:14.488491Z ResponseDesc=Created Exiting. 2022-04-26T06:24:14.489Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:24:14.501Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=563b17ad-7ef1-46be-9ec8-8fb8795d06e2 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:14.501166Z InvocationID=aa026a6e-30de-4cd5-bd3b-c47bfc084829 LogTimestamp=2022-04-26T06:24:14.501234Z Entering 2022-04-26T06:24:14.501Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=563b17ad-7ef1-46be-9ec8-8fb8795d06e2 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:14.501166Z InvocationID=aa026a6e-30de-4cd5-bd3b-c47bfc084829 LogTimestamp=2022-04-26T06:24:14.501234Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:24:14.623Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:14.502Z InvocationID=f1227675-96ba-44c8-8fa9-bae842d3c306 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:14.623Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:14.502Z InvocationID=f1227675-96ba-44c8-8fa9-bae842d3c306 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:14.624Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:14.502Z InvocationID=f1227675-96ba-44c8-8fa9-bae842d3c306 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:14.624Z [qtp2085002312-13] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:14.502Z InvocationID=f1227675-96ba-44c8-8fa9-bae842d3c306 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:14.657Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:14.502Z InvocationID=f1227675-96ba-44c8-8fa9-bae842d3c306 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name ExtZteCp 2022-04-26T06:24:14.707Z [qtp2085002312-13] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ext ZTE CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "8d336cea-f23f-4e32-a7e8-1b71bd02da7e" INVARIANT_UUID = "bd6f06bb-7bab-413b-a675-b7a9fa792ddf" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.zte.CP" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8d336cea-f23f-4e32-a7e8-1b71bd02da7e EntryTimestamp=2022-04-26T06:24:14.502Z InvocationID=f1227675-96ba-44c8-8fa9-bae842d3c306 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:24:14.708Z [qtp2085002312-13] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ext ZTE CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "8d336cea-f23f-4e32-a7e8-1b71bd02da7e" INVARIANT_UUID = "bd6f06bb-7bab-413b-a675-b7a9fa792ddf" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.zte.CP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8d336cea-f23f-4e32-a7e8-1b71bd02da7e TargetServiceName=null EntryTimestamp=2022-04-26T06:24:14.502Z InvocationID=f1227675-96ba-44c8-8fa9-bae842d3c306 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:24:14.754Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ext ZTE CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "8d336cea-f23f-4e32-a7e8-1b71bd02da7e" INVARIANT_UUID = "bd6f06bb-7bab-413b-a675-b7a9fa792ddf" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.zte.CP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8d336cea-f23f-4e32-a7e8-1b71bd02da7e TargetServiceName=null EntryTimestamp=2022-04-26T06:24:14.502Z InvocationID=f1227675-96ba-44c8-8fa9-bae842d3c306 ErrorCode=500 lock resource with id 1eae0e19-331e-4ee0-9419-0152295c8f13 2022-04-26T06:24:15.363Z [qtp2085002312-13] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ext ZTE CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "8d336cea-f23f-4e32-a7e8-1b71bd02da7e" INVARIANT_UUID = "bd6f06bb-7bab-413b-a675-b7a9fa792ddf" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1eae0e19-331e-4ee0-9419-0152295c8f13.assettoscatemplate" CURR_ARTIFACT_UUID = "1eae0e19-331e-4ee0-9419-0152295c8f13.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtZteCp-template.yml,0,1eae0e19-331e-4ee0-9419-0152295c8f13.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1b90e2a7-70ce-44dd-9fd5-c0c3587a35f2 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:14.502Z InvocationID=f1227675-96ba-44c8-8fa9-bae842d3c306 ErrorCode=500 Starting to create Csar for component ext ZTE CP 2022-04-26T06:24:15.513Z [qtp2085002312-13] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ext ZTE CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "8d336cea-f23f-4e32-a7e8-1b71bd02da7e" INVARIANT_UUID = "bd6f06bb-7bab-413b-a675-b7a9fa792ddf" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1eae0e19-331e-4ee0-9419-0152295c8f13.assettoscatemplate" CURR_ARTIFACT_UUID = "1eae0e19-331e-4ee0-9419-0152295c8f13.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtZteCp-template.yml,0,1eae0e19-331e-4ee0-9419-0152295c8f13.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1b90e2a7-70ce-44dd-9fd5-c0c3587a35f2 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:14.502Z InvocationID=f1227675-96ba-44c8-8fa9-bae842d3c306 ErrorCode=500 Ended create Csar for component ext ZTE CP 2022-04-26T06:24:15.514Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ext ZTE CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "8d336cea-f23f-4e32-a7e8-1b71bd02da7e" INVARIANT_UUID = "bd6f06bb-7bab-413b-a675-b7a9fa792ddf" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1eae0e19-331e-4ee0-9419-0152295c8f13.assettoscatemplate" CURR_ARTIFACT_UUID = "1eae0e19-331e-4ee0-9419-0152295c8f13.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtZteCp-template.yml,0,1eae0e19-331e-4ee0-9419-0152295c8f13.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1b90e2a7-70ce-44dd-9fd5-c0c3587a35f2 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:14.502Z InvocationID=f1227675-96ba-44c8-8fa9-bae842d3c306 ErrorCode=500 lock resource with id 1eae0e19-331e-4ee0-9419-0152295c8f13 2022-04-26T06:24:15.840Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ef0269e-de2c-4ddc-9f4b-f2de2b2fb918 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:24:15.840Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ef0269e-de2c-4ddc-9f4b-f2de2b2fb918 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5c8b8ca2 2022-04-26T06:24:15.841Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ef0269e-de2c-4ddc-9f4b-f2de2b2fb918 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:24:15.841Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ef0269e-de2c-4ddc-9f4b-f2de2b2fb918 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:24:16.071Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ext ZTE CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "8d336cea-f23f-4e32-a7e8-1b71bd02da7e" INVARIANT_UUID = "bd6f06bb-7bab-413b-a675-b7a9fa792ddf" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1eae0e19-331e-4ee0-9419-0152295c8f13.assettoscacsar" CURR_ARTIFACT_UUID = "1eae0e19-331e-4ee0-9419-0152295c8f13.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-ExtZteCp-csar.csar,0,1eae0e19-331e-4ee0-9419-0152295c8f13.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1569 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b80cf231-9fca-4646-bab4-5236f4147cd9 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:14.502Z InvocationID=f1227675-96ba-44c8-8fa9-bae842d3c306 ErrorCode=500 LogTimestamp=2022-04-26T06:24:16.071427Z ResponseDesc=Created Exiting. 2022-04-26T06:24:16.072Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:24:16.083Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=d2eff573-1023-4ae1-8020-b9f84714dc01 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:16.083462Z InvocationID=a5b8871d-ed3d-4900-9c22-9b2079ee747e LogTimestamp=2022-04-26T06:24:16.083521Z Entering 2022-04-26T06:24:16.083Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=d2eff573-1023-4ae1-8020-b9f84714dc01 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:16.083462Z InvocationID=a5b8871d-ed3d-4900-9c22-9b2079ee747e LogTimestamp=2022-04-26T06:24:16.083521Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:24:16.203Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:16.084Z InvocationID=90190bf6-cc27-4f40-b71b-f33e464bcdc4 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:16.203Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:16.084Z InvocationID=90190bf6-cc27-4f40-b71b-f33e464bcdc4 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:16.203Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:16.084Z InvocationID=90190bf6-cc27-4f40-b71b-f33e464bcdc4 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:16.203Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:16.084Z InvocationID=90190bf6-cc27-4f40-b71b-f33e464bcdc4 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:16.291Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:16.084Z InvocationID=90190bf6-cc27-4f40-b71b-f33e464bcdc4 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name ExtZteVdu 2022-04-26T06:24:16.335Z [qtp2085002312-15] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Ext Zte VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "b49ad9f2-165e-4b26-a265-4da557c18c26" INVARIANT_UUID = "bb4c9ff5-72b0-4220-a72f-69733d804072" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.zte.VDU" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b49ad9f2-165e-4b26-a265-4da557c18c26 EntryTimestamp=2022-04-26T06:24:16.084Z InvocationID=90190bf6-cc27-4f40-b71b-f33e464bcdc4 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:24:16.335Z [qtp2085002312-15] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Ext Zte VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "b49ad9f2-165e-4b26-a265-4da557c18c26" INVARIANT_UUID = "bb4c9ff5-72b0-4220-a72f-69733d804072" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.zte.VDU" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b49ad9f2-165e-4b26-a265-4da557c18c26 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:16.084Z InvocationID=90190bf6-cc27-4f40-b71b-f33e464bcdc4 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:24:16.377Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Ext Zte VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "b49ad9f2-165e-4b26-a265-4da557c18c26" INVARIANT_UUID = "bb4c9ff5-72b0-4220-a72f-69733d804072" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.zte.VDU" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b49ad9f2-165e-4b26-a265-4da557c18c26 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:16.084Z InvocationID=90190bf6-cc27-4f40-b71b-f33e464bcdc4 ErrorCode=500 lock resource with id bb4c1a8b-3fc2-486b-80e4-0e9e20238bff 2022-04-26T06:24:16.873Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=5ef0269e-de2c-4ddc-9f4b-f2de2b2fb918 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:24:16.873Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=5ef0269e-de2c-4ddc-9f4b-f2de2b2fb918 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:24:17.048Z [qtp2085002312-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Ext Zte VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "b49ad9f2-165e-4b26-a265-4da557c18c26" INVARIANT_UUID = "bb4c9ff5-72b0-4220-a72f-69733d804072" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "bb4c1a8b-3fc2-486b-80e4-0e9e20238bff.assettoscatemplate" CURR_ARTIFACT_UUID = "bb4c1a8b-3fc2-486b-80e4-0e9e20238bff.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtZteVdu-template.yml,0,bb4c1a8b-3fc2-486b-80e4-0e9e20238bff.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2b89f4dd-de03-4ceb-ba3d-78102ac7262c TargetServiceName=null EntryTimestamp=2022-04-26T06:24:16.084Z InvocationID=90190bf6-cc27-4f40-b71b-f33e464bcdc4 ErrorCode=500 Starting to create Csar for component Ext Zte VDU 2022-04-26T06:24:17.198Z [qtp2085002312-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Ext Zte VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "b49ad9f2-165e-4b26-a265-4da557c18c26" INVARIANT_UUID = "bb4c9ff5-72b0-4220-a72f-69733d804072" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "bb4c1a8b-3fc2-486b-80e4-0e9e20238bff.assettoscatemplate" CURR_ARTIFACT_UUID = "bb4c1a8b-3fc2-486b-80e4-0e9e20238bff.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtZteVdu-template.yml,0,bb4c1a8b-3fc2-486b-80e4-0e9e20238bff.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2b89f4dd-de03-4ceb-ba3d-78102ac7262c TargetServiceName=null EntryTimestamp=2022-04-26T06:24:16.084Z InvocationID=90190bf6-cc27-4f40-b71b-f33e464bcdc4 ErrorCode=500 Ended create Csar for component Ext Zte VDU 2022-04-26T06:24:17.199Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Ext Zte VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "b49ad9f2-165e-4b26-a265-4da557c18c26" INVARIANT_UUID = "bb4c9ff5-72b0-4220-a72f-69733d804072" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "bb4c1a8b-3fc2-486b-80e4-0e9e20238bff.assettoscatemplate" CURR_ARTIFACT_UUID = "bb4c1a8b-3fc2-486b-80e4-0e9e20238bff.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtZteVdu-template.yml,0,bb4c1a8b-3fc2-486b-80e4-0e9e20238bff.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2b89f4dd-de03-4ceb-ba3d-78102ac7262c TargetServiceName=null EntryTimestamp=2022-04-26T06:24:16.084Z InvocationID=90190bf6-cc27-4f40-b71b-f33e464bcdc4 ErrorCode=500 lock resource with id bb4c1a8b-3fc2-486b-80e4-0e9e20238bff 2022-04-26T06:24:17.768Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Ext Zte VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "b49ad9f2-165e-4b26-a265-4da557c18c26" INVARIANT_UUID = "bb4c9ff5-72b0-4220-a72f-69733d804072" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "bb4c1a8b-3fc2-486b-80e4-0e9e20238bff.assettoscacsar" CURR_ARTIFACT_UUID = "bb4c1a8b-3fc2-486b-80e4-0e9e20238bff.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-ExtZteVdu-csar.csar,0,bb4c1a8b-3fc2-486b-80e4-0e9e20238bff.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1684 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3c84d7f8-28f4-4c75-bf38-d5abea81d466 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:16.084Z InvocationID=90190bf6-cc27-4f40-b71b-f33e464bcdc4 ErrorCode=500 LogTimestamp=2022-04-26T06:24:17.768060Z ResponseDesc=Created Exiting. 2022-04-26T06:24:17.768Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:24:17.782Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=3721d2e9-03b8-4cf2-9d58-c69d52ddc14b InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:17.782346Z InvocationID=672f6dc9-2ffe-4e3c-a2ab-9886dd5217b9 LogTimestamp=2022-04-26T06:24:17.782447Z Entering 2022-04-26T06:24:17.783Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=3721d2e9-03b8-4cf2-9d58-c69d52ddc14b InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:17.782346Z InvocationID=672f6dc9-2ffe-4e3c-a2ab-9886dd5217b9 LogTimestamp=2022-04-26T06:24:17.782447Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:24:17.953Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:17.783Z InvocationID=a5b7c48f-3848-4f08-9b8e-e115e7b3ee41 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:17.953Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:17.783Z InvocationID=a5b7c48f-3848-4f08-9b8e-e115e7b3ee41 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:17.953Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:17.783Z InvocationID=a5b7c48f-3848-4f08-9b8e-e115e7b3ee41 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:17.954Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:17.783Z InvocationID=a5b7c48f-3848-4f08-9b8e-e115e7b3ee41 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:17.958Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:17.783Z InvocationID=a5b7c48f-3848-4f08-9b8e-e115e7b3ee41 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name ExtZteVl 2022-04-26T06:24:18.008Z [qtp2085002312-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ext ZTE VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "29f5fe01-11ed-4926-918e-36e1a2a51ed7" INVARIANT_UUID = "48dd586b-d0d5-4eb9-a722-0080f0ab54bd" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.zte.VL" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=29f5fe01-11ed-4926-918e-36e1a2a51ed7 EntryTimestamp=2022-04-26T06:24:17.783Z InvocationID=a5b7c48f-3848-4f08-9b8e-e115e7b3ee41 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:24:18.009Z [qtp2085002312-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ext ZTE VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "29f5fe01-11ed-4926-918e-36e1a2a51ed7" INVARIANT_UUID = "48dd586b-d0d5-4eb9-a722-0080f0ab54bd" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.zte.VL" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=29f5fe01-11ed-4926-918e-36e1a2a51ed7 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:17.783Z InvocationID=a5b7c48f-3848-4f08-9b8e-e115e7b3ee41 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:24:18.054Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ext ZTE VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "29f5fe01-11ed-4926-918e-36e1a2a51ed7" INVARIANT_UUID = "48dd586b-d0d5-4eb9-a722-0080f0ab54bd" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.zte.VL" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=29f5fe01-11ed-4926-918e-36e1a2a51ed7 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:17.783Z InvocationID=a5b7c48f-3848-4f08-9b8e-e115e7b3ee41 ErrorCode=500 lock resource with id 5c66ac8b-1b87-4202-b0e2-c9c407d511b5 2022-04-26T06:24:18.840Z [qtp2085002312-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ext ZTE VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "29f5fe01-11ed-4926-918e-36e1a2a51ed7" INVARIANT_UUID = "48dd586b-d0d5-4eb9-a722-0080f0ab54bd" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5c66ac8b-1b87-4202-b0e2-c9c407d511b5.assettoscatemplate" CURR_ARTIFACT_UUID = "5c66ac8b-1b87-4202-b0e2-c9c407d511b5.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtZteVl-template.yml,0,5c66ac8b-1b87-4202-b0e2-c9c407d511b5.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=312625a8-4fd7-4b11-8616-8c61aa139461 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:17.783Z InvocationID=a5b7c48f-3848-4f08-9b8e-e115e7b3ee41 ErrorCode=500 Starting to create Csar for component ext ZTE VL 2022-04-26T06:24:18.895Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60666782-8f02-4db6-b169-3162b3ce2bc1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:24:18.896Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60666782-8f02-4db6-b169-3162b3ce2bc1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f1a7726 2022-04-26T06:24:18.896Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60666782-8f02-4db6-b169-3162b3ce2bc1 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:24:18.896Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60666782-8f02-4db6-b169-3162b3ce2bc1 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:24:19.015Z [qtp2085002312-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ext ZTE VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "29f5fe01-11ed-4926-918e-36e1a2a51ed7" INVARIANT_UUID = "48dd586b-d0d5-4eb9-a722-0080f0ab54bd" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5c66ac8b-1b87-4202-b0e2-c9c407d511b5.assettoscatemplate" CURR_ARTIFACT_UUID = "5c66ac8b-1b87-4202-b0e2-c9c407d511b5.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtZteVl-template.yml,0,5c66ac8b-1b87-4202-b0e2-c9c407d511b5.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=312625a8-4fd7-4b11-8616-8c61aa139461 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:17.783Z InvocationID=a5b7c48f-3848-4f08-9b8e-e115e7b3ee41 ErrorCode=500 Ended create Csar for component ext ZTE VL 2022-04-26T06:24:19.015Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ext ZTE VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "29f5fe01-11ed-4926-918e-36e1a2a51ed7" INVARIANT_UUID = "48dd586b-d0d5-4eb9-a722-0080f0ab54bd" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5c66ac8b-1b87-4202-b0e2-c9c407d511b5.assettoscatemplate" CURR_ARTIFACT_UUID = "5c66ac8b-1b87-4202-b0e2-c9c407d511b5.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtZteVl-template.yml,0,5c66ac8b-1b87-4202-b0e2-c9c407d511b5.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=312625a8-4fd7-4b11-8616-8c61aa139461 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:17.783Z InvocationID=a5b7c48f-3848-4f08-9b8e-e115e7b3ee41 ErrorCode=500 lock resource with id 5c66ac8b-1b87-4202-b0e2-c9c407d511b5 2022-04-26T06:24:19.568Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ext ZTE VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "29f5fe01-11ed-4926-918e-36e1a2a51ed7" INVARIANT_UUID = "48dd586b-d0d5-4eb9-a722-0080f0ab54bd" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5c66ac8b-1b87-4202-b0e2-c9c407d511b5.assettoscacsar" CURR_ARTIFACT_UUID = "5c66ac8b-1b87-4202-b0e2-c9c407d511b5.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-ExtZteVl-csar.csar,0,5c66ac8b-1b87-4202-b0e2-c9c407d511b5.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1784 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f09df6da-fe41-421f-9b3e-e2b3fcbdfc69 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:17.783Z InvocationID=a5b7c48f-3848-4f08-9b8e-e115e7b3ee41 ErrorCode=500 LogTimestamp=2022-04-26T06:24:19.567843Z ResponseDesc=Created Exiting. 2022-04-26T06:24:19.568Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:24:19.644Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1fa440d3-0dfa-4161-9266-90f8c911ddad InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:19.643689Z InvocationID=d14dbb03-305b-4e1a-91d8-6a709afbb0e9 LogTimestamp=2022-04-26T06:24:19.643777Z Entering 2022-04-26T06:24:19.644Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1fa440d3-0dfa-4161-9266-90f8c911ddad InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:19.643689Z InvocationID=d14dbb03-305b-4e1a-91d8-6a709afbb0e9 LogTimestamp=2022-04-26T06:24:19.643777Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:24:19.914Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=60666782-8f02-4db6-b169-3162b3ce2bc1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:24:19.914Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=60666782-8f02-4db6-b169-3162b3ce2bc1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:24:19.965Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:19.644Z InvocationID=d867e4c5-2fc2-40a9-a792-f52bb2595943 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:19.965Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:19.644Z InvocationID=d867e4c5-2fc2-40a9-a792-f52bb2595943 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:19.966Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:19.644Z InvocationID=d867e4c5-2fc2-40a9-a792-f52bb2595943 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:19.966Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:19.644Z InvocationID=d867e4c5-2fc2-40a9-a792-f52bb2595943 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:19.971Z [qtp2085002312-12] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Create Resource - validateLifecycleTypesCreate ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:24:19.644Z InvocationID=d867e4c5-2fc2-40a9-a792-f52bb2595943 ErrorCode=400 Interface tosca.interfaces.nfv.nslcm required is missing 2022-04-26T06:24:19.972Z [qtp2085002312-12] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Create Resource - validateLifecycleTypesCreate ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:24:19.644Z InvocationID=d867e4c5-2fc2-40a9-a792-f52bb2595943 ErrorCode=500 Operation towards database failed. 2022-04-26T06:24:19.972Z [qtp2085002312-12] WARN o.o.s.e.AbstractSdncException RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:24:19.644Z InvocationID=d867e4c5-2fc2-40a9-a792-f52bb2595943 ErrorCode=900 2022-04-26T06:24:19.976Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "" INVARIANT_UUID = "" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "400" DESC = "SVC4134: Error: Invalid Content. Missing interface life-cycle type." TOSCA_NODE_TYPE = "tosca.nodes.nfv.NS" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:24:19.644Z InvocationID=d867e4c5-2fc2-40a9-a792-f52bb2595943 ErrorCode=900 lock resource with name Ns 2022-04-26T06:24:20.148Z [qtp2085002312-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d3c4a439-f267-4b7f-8981-81ea696b8dfd" INVARIANT_UUID = "d072405b-9c53-4793-9678-68d02bed3d8a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NS" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d3c4a439-f267-4b7f-8981-81ea696b8dfd TargetServiceName=null EntryTimestamp=2022-04-26T06:24:19.644Z InvocationID=d867e4c5-2fc2-40a9-a792-f52bb2595943 ErrorCode=900 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:24:20.149Z [qtp2085002312-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d3c4a439-f267-4b7f-8981-81ea696b8dfd" INVARIANT_UUID = "d072405b-9c53-4793-9678-68d02bed3d8a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NS" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d3c4a439-f267-4b7f-8981-81ea696b8dfd TargetServiceName=null EntryTimestamp=2022-04-26T06:24:19.644Z InvocationID=d867e4c5-2fc2-40a9-a792-f52bb2595943 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:24:20.265Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d3c4a439-f267-4b7f-8981-81ea696b8dfd" INVARIANT_UUID = "d072405b-9c53-4793-9678-68d02bed3d8a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NS" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d3c4a439-f267-4b7f-8981-81ea696b8dfd TargetServiceName=null EntryTimestamp=2022-04-26T06:24:19.644Z InvocationID=d867e4c5-2fc2-40a9-a792-f52bb2595943 ErrorCode=500 lock resource with id 36b3fccc-7cea-4cd3-b2e9-6d11cb572122 2022-04-26T06:24:20.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:24:20.374Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:24:21.353Z [qtp2085002312-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d3c4a439-f267-4b7f-8981-81ea696b8dfd" INVARIANT_UUID = "d072405b-9c53-4793-9678-68d02bed3d8a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "36b3fccc-7cea-4cd3-b2e9-6d11cb572122.assettoscatemplate" CURR_ARTIFACT_UUID = "36b3fccc-7cea-4cd3-b2e9-6d11cb572122.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ns-template.yml,0,36b3fccc-7cea-4cd3-b2e9-6d11cb572122.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e8e11789-ad93-488c-a85c-e58fa70995c8 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:19.644Z InvocationID=d867e4c5-2fc2-40a9-a792-f52bb2595943 ErrorCode=500 Starting to create Csar for component NS 2022-04-26T06:24:21.513Z [qtp2085002312-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d3c4a439-f267-4b7f-8981-81ea696b8dfd" INVARIANT_UUID = "d072405b-9c53-4793-9678-68d02bed3d8a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "36b3fccc-7cea-4cd3-b2e9-6d11cb572122.assettoscatemplate" CURR_ARTIFACT_UUID = "36b3fccc-7cea-4cd3-b2e9-6d11cb572122.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ns-template.yml,0,36b3fccc-7cea-4cd3-b2e9-6d11cb572122.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e8e11789-ad93-488c-a85c-e58fa70995c8 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:19.644Z InvocationID=d867e4c5-2fc2-40a9-a792-f52bb2595943 ErrorCode=500 Ended create Csar for component NS 2022-04-26T06:24:21.514Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d3c4a439-f267-4b7f-8981-81ea696b8dfd" INVARIANT_UUID = "d072405b-9c53-4793-9678-68d02bed3d8a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "36b3fccc-7cea-4cd3-b2e9-6d11cb572122.assettoscatemplate" CURR_ARTIFACT_UUID = "36b3fccc-7cea-4cd3-b2e9-6d11cb572122.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ns-template.yml,0,36b3fccc-7cea-4cd3-b2e9-6d11cb572122.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e8e11789-ad93-488c-a85c-e58fa70995c8 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:19.644Z InvocationID=d867e4c5-2fc2-40a9-a792-f52bb2595943 ErrorCode=500 lock resource with id 36b3fccc-7cea-4cd3-b2e9-6d11cb572122 2022-04-26T06:24:22.365Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d3c4a439-f267-4b7f-8981-81ea696b8dfd" INVARIANT_UUID = "d072405b-9c53-4793-9678-68d02bed3d8a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "36b3fccc-7cea-4cd3-b2e9-6d11cb572122.assettoscacsar" CURR_ARTIFACT_UUID = "36b3fccc-7cea-4cd3-b2e9-6d11cb572122.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Ns-csar.csar,0,36b3fccc-7cea-4cd3-b2e9-6d11cb572122.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=2721 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b2422970-1fa9-4a69-a5e6-b8f79f534598 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:19.644Z InvocationID=d867e4c5-2fc2-40a9-a792-f52bb2595943 ErrorCode=500 LogTimestamp=2022-04-26T06:24:22.365141Z ResponseDesc=Created Exiting. 2022-04-26T06:24:22.368Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:24:22.380Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=b8fb9e31-9e01-4986-a236-ead941c52feb InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:22.380549Z InvocationID=b2e1fa03-062c-46ed-a238-2df96fbc3f8b LogTimestamp=2022-04-26T06:24:22.380620Z Entering 2022-04-26T06:24:22.381Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=b8fb9e31-9e01-4986-a236-ead941c52feb InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:22.380549Z InvocationID=b2e1fa03-062c-46ed-a238-2df96fbc3f8b LogTimestamp=2022-04-26T06:24:22.380620Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:24:22.746Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b08455ee-4732-40bb-899e-3430aef0a850 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:24:22.746Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b08455ee-4732-40bb-899e-3430aef0a850 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@668bfd91 2022-04-26T06:24:22.746Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b08455ee-4732-40bb-899e-3430aef0a850 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:24:22.746Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b08455ee-4732-40bb-899e-3430aef0a850 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:24:23.063Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:22.381Z InvocationID=25e0a115-b5bc-4d9c-9c4f-2dde75e0ce20 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:23.063Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:22.381Z InvocationID=25e0a115-b5bc-4d9c-9c4f-2dde75e0ce20 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:23.063Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:22.381Z InvocationID=25e0a115-b5bc-4d9c-9c4f-2dde75e0ce20 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:23.063Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:22.381Z InvocationID=25e0a115-b5bc-4d9c-9c4f-2dde75e0ce20 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:23.146Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:22.381Z InvocationID=25e0a115-b5bc-4d9c-9c4f-2dde75e0ce20 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Nsd 2022-04-26T06:24:23.444Z [qtp2085002312-15] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NSD" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "72864b1f-1adb-4bb9-a68b-7790ea13efb9" INVARIANT_UUID = "5a5d2ece-d0be-4adb-b41a-2a6e66fda6e8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.NSD" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=72864b1f-1adb-4bb9-a68b-7790ea13efb9 EntryTimestamp=2022-04-26T06:24:22.381Z InvocationID=25e0a115-b5bc-4d9c-9c4f-2dde75e0ce20 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:24:23.444Z [qtp2085002312-15] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NSD" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "72864b1f-1adb-4bb9-a68b-7790ea13efb9" INVARIANT_UUID = "5a5d2ece-d0be-4adb-b41a-2a6e66fda6e8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.NSD" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=72864b1f-1adb-4bb9-a68b-7790ea13efb9 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:22.381Z InvocationID=25e0a115-b5bc-4d9c-9c4f-2dde75e0ce20 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:24:23.763Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NSD" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "72864b1f-1adb-4bb9-a68b-7790ea13efb9" INVARIANT_UUID = "5a5d2ece-d0be-4adb-b41a-2a6e66fda6e8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.NSD" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=72864b1f-1adb-4bb9-a68b-7790ea13efb9 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:22.381Z InvocationID=25e0a115-b5bc-4d9c-9c4f-2dde75e0ce20 ErrorCode=500 lock resource with id 03248ce7-6d5b-45bc-8e9e-054e63efc78d 2022-04-26T06:24:23.785Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=b08455ee-4732-40bb-899e-3430aef0a850 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:24:23.785Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=b08455ee-4732-40bb-899e-3430aef0a850 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:24:25.452Z [qtp2085002312-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NSD" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "72864b1f-1adb-4bb9-a68b-7790ea13efb9" INVARIANT_UUID = "5a5d2ece-d0be-4adb-b41a-2a6e66fda6e8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "03248ce7-6d5b-45bc-8e9e-054e63efc78d.assettoscatemplate" CURR_ARTIFACT_UUID = "03248ce7-6d5b-45bc-8e9e-054e63efc78d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nsd-template.yml,0,03248ce7-6d5b-45bc-8e9e-054e63efc78d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b36962be-69ce-4356-b1de-79c801835ab4 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:22.381Z InvocationID=25e0a115-b5bc-4d9c-9c4f-2dde75e0ce20 ErrorCode=500 Starting to create Csar for component NSD 2022-04-26T06:24:25.705Z [qtp2085002312-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NSD" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "72864b1f-1adb-4bb9-a68b-7790ea13efb9" INVARIANT_UUID = "5a5d2ece-d0be-4adb-b41a-2a6e66fda6e8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "03248ce7-6d5b-45bc-8e9e-054e63efc78d.assettoscatemplate" CURR_ARTIFACT_UUID = "03248ce7-6d5b-45bc-8e9e-054e63efc78d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nsd-template.yml,0,03248ce7-6d5b-45bc-8e9e-054e63efc78d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b36962be-69ce-4356-b1de-79c801835ab4 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:22.381Z InvocationID=25e0a115-b5bc-4d9c-9c4f-2dde75e0ce20 ErrorCode=500 Ended create Csar for component NSD 2022-04-26T06:24:25.706Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NSD" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "72864b1f-1adb-4bb9-a68b-7790ea13efb9" INVARIANT_UUID = "5a5d2ece-d0be-4adb-b41a-2a6e66fda6e8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "03248ce7-6d5b-45bc-8e9e-054e63efc78d.assettoscatemplate" CURR_ARTIFACT_UUID = "03248ce7-6d5b-45bc-8e9e-054e63efc78d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nsd-template.yml,0,03248ce7-6d5b-45bc-8e9e-054e63efc78d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b36962be-69ce-4356-b1de-79c801835ab4 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:22.381Z InvocationID=25e0a115-b5bc-4d9c-9c4f-2dde75e0ce20 ErrorCode=500 lock resource with id 03248ce7-6d5b-45bc-8e9e-054e63efc78d 2022-04-26T06:24:25.845Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eb809e44-89b2-4a57-954c-6414d2d990f4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:24:25.846Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eb809e44-89b2-4a57-954c-6414d2d990f4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@271805d3 2022-04-26T06:24:25.846Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eb809e44-89b2-4a57-954c-6414d2d990f4 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:24:25.846Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eb809e44-89b2-4a57-954c-6414d2d990f4 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:24:26.555Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NSD" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "72864b1f-1adb-4bb9-a68b-7790ea13efb9" INVARIANT_UUID = "5a5d2ece-d0be-4adb-b41a-2a6e66fda6e8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "03248ce7-6d5b-45bc-8e9e-054e63efc78d.assettoscacsar" CURR_ARTIFACT_UUID = "03248ce7-6d5b-45bc-8e9e-054e63efc78d.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Nsd-csar.csar,0,03248ce7-6d5b-45bc-8e9e-054e63efc78d.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=4174 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2f3b679e-8b31-4a46-9216-13c09c02d751 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:22.381Z InvocationID=25e0a115-b5bc-4d9c-9c4f-2dde75e0ce20 ErrorCode=500 LogTimestamp=2022-04-26T06:24:26.555599Z ResponseDesc=Created Exiting. 2022-04-26T06:24:26.556Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:24:26.569Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=d912a2b1-5976-4cbf-a5fe-c388f2f782c7 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:26.569076Z InvocationID=5694065c-f8ea-463c-b4f4-a9c041b70d63 LogTimestamp=2022-04-26T06:24:26.569147Z Entering 2022-04-26T06:24:26.569Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=d912a2b1-5976-4cbf-a5fe-c388f2f782c7 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:26.569076Z InvocationID=5694065c-f8ea-463c-b4f4-a9c041b70d63 LogTimestamp=2022-04-26T06:24:26.569147Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:24:26.860Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=eb809e44-89b2-4a57-954c-6414d2d990f4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:24:26.861Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=eb809e44-89b2-4a57-954c-6414d2d990f4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:24:27.256Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:26.570Z InvocationID=e5307229-738d-4834-b382-5a29e11649ee PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:27.257Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:26.570Z InvocationID=e5307229-738d-4834-b382-5a29e11649ee PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:27.257Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:26.570Z InvocationID=e5307229-738d-4834-b382-5a29e11649ee PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:27.257Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:26.570Z InvocationID=e5307229-738d-4834-b382-5a29e11649ee PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:27.341Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:26.570Z InvocationID=e5307229-738d-4834-b382-5a29e11649ee PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Nsvirtuallink 2022-04-26T06:24:27.548Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NsVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "7fdaaac6-81c7-4e2d-a65e-b8b6fc1a92b4" INVARIANT_UUID = "227ef74a-6abf-406c-b2a1-d9df8a21546b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NsVirtualLink" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7fdaaac6-81c7-4e2d-a65e-b8b6fc1a92b4 EntryTimestamp=2022-04-26T06:24:26.570Z InvocationID=e5307229-738d-4834-b382-5a29e11649ee PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:24:27.548Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NsVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "7fdaaac6-81c7-4e2d-a65e-b8b6fc1a92b4" INVARIANT_UUID = "227ef74a-6abf-406c-b2a1-d9df8a21546b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NsVirtualLink" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7fdaaac6-81c7-4e2d-a65e-b8b6fc1a92b4 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:26.570Z InvocationID=e5307229-738d-4834-b382-5a29e11649ee ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:24:27.949Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NsVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "7fdaaac6-81c7-4e2d-a65e-b8b6fc1a92b4" INVARIANT_UUID = "227ef74a-6abf-406c-b2a1-d9df8a21546b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NsVirtualLink" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7fdaaac6-81c7-4e2d-a65e-b8b6fc1a92b4 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:26.570Z InvocationID=e5307229-738d-4834-b382-5a29e11649ee ErrorCode=500 lock resource with id b3d1cbfc-cd00-4b02-8447-7cecb99abeab 2022-04-26T06:24:28.549Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3163bda1-59a6-4ae8-93c0-3c66a54bb2cd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:24:28.549Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3163bda1-59a6-4ae8-93c0-3c66a54bb2cd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@54f27b99 2022-04-26T06:24:28.549Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3163bda1-59a6-4ae8-93c0-3c66a54bb2cd PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:24:28.550Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3163bda1-59a6-4ae8-93c0-3c66a54bb2cd PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:24:29.546Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NsVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "7fdaaac6-81c7-4e2d-a65e-b8b6fc1a92b4" INVARIANT_UUID = "227ef74a-6abf-406c-b2a1-d9df8a21546b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b3d1cbfc-cd00-4b02-8447-7cecb99abeab.assettoscatemplate" CURR_ARTIFACT_UUID = "b3d1cbfc-cd00-4b02-8447-7cecb99abeab.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nsvirtuallink-template.yml,0,b3d1cbfc-cd00-4b02-8447-7cecb99abeab.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=056b1a01-ccf9-4c86-99b8-ac0ed6dd4185 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:26.570Z InvocationID=e5307229-738d-4834-b382-5a29e11649ee ErrorCode=500 Starting to create Csar for component NsVirtualLink 2022-04-26T06:24:29.577Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3163bda1-59a6-4ae8-93c0-3c66a54bb2cd ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:24:29.577Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3163bda1-59a6-4ae8-93c0-3c66a54bb2cd ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:24:29.802Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NsVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "7fdaaac6-81c7-4e2d-a65e-b8b6fc1a92b4" INVARIANT_UUID = "227ef74a-6abf-406c-b2a1-d9df8a21546b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b3d1cbfc-cd00-4b02-8447-7cecb99abeab.assettoscatemplate" CURR_ARTIFACT_UUID = "b3d1cbfc-cd00-4b02-8447-7cecb99abeab.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nsvirtuallink-template.yml,0,b3d1cbfc-cd00-4b02-8447-7cecb99abeab.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=056b1a01-ccf9-4c86-99b8-ac0ed6dd4185 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:26.570Z InvocationID=e5307229-738d-4834-b382-5a29e11649ee ErrorCode=500 Ended create Csar for component NsVirtualLink 2022-04-26T06:24:29.802Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NsVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "7fdaaac6-81c7-4e2d-a65e-b8b6fc1a92b4" INVARIANT_UUID = "227ef74a-6abf-406c-b2a1-d9df8a21546b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b3d1cbfc-cd00-4b02-8447-7cecb99abeab.assettoscatemplate" CURR_ARTIFACT_UUID = "b3d1cbfc-cd00-4b02-8447-7cecb99abeab.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nsvirtuallink-template.yml,0,b3d1cbfc-cd00-4b02-8447-7cecb99abeab.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=056b1a01-ccf9-4c86-99b8-ac0ed6dd4185 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:26.570Z InvocationID=e5307229-738d-4834-b382-5a29e11649ee ErrorCode=500 lock resource with id b3d1cbfc-cd00-4b02-8447-7cecb99abeab 2022-04-26T06:24:31.352Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NsVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "7fdaaac6-81c7-4e2d-a65e-b8b6fc1a92b4" INVARIANT_UUID = "227ef74a-6abf-406c-b2a1-d9df8a21546b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b3d1cbfc-cd00-4b02-8447-7cecb99abeab.assettoscacsar" CURR_ARTIFACT_UUID = "b3d1cbfc-cd00-4b02-8447-7cecb99abeab.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Nsvirtuallink-csar.csar,0,b3d1cbfc-cd00-4b02-8447-7cecb99abeab.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=4782 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0cf7a354-6e4f-429c-8398-88009eb8ab97 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:26.570Z InvocationID=e5307229-738d-4834-b382-5a29e11649ee ErrorCode=500 LogTimestamp=2022-04-26T06:24:31.352309Z ResponseDesc=Created Exiting. 2022-04-26T06:24:31.353Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:24:31.434Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=2e33090a-85b3-42f5-9e9f-7d14e6f8ec93 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:31.433979Z InvocationID=014483ca-3297-412a-bcad-e299e6b7570c LogTimestamp=2022-04-26T06:24:31.434052Z Entering 2022-04-26T06:24:31.434Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=2e33090a-85b3-42f5-9e9f-7d14e6f8ec93 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:31.433979Z InvocationID=014483ca-3297-412a-bcad-e299e6b7570c LogTimestamp=2022-04-26T06:24:31.434052Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:24:31.634Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae9c3175-0a99-4f7a-b020-ac41c2f6e1fb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:24:31.634Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae9c3175-0a99-4f7a-b020-ac41c2f6e1fb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@78637e3 2022-04-26T06:24:31.634Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae9c3175-0a99-4f7a-b020-ac41c2f6e1fb PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:24:31.634Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae9c3175-0a99-4f7a-b020-ac41c2f6e1fb PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:24:32.681Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ae9c3175-0a99-4f7a-b020-ac41c2f6e1fb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:24:32.681Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ae9c3175-0a99-4f7a-b020-ac41c2f6e1fb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:24:33.242Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:31.435Z InvocationID=a6104310-f554-4ea5-ad93-fc47380207c8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:33.243Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:31.435Z InvocationID=a6104310-f554-4ea5-ad93-fc47380207c8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:33.243Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:31.435Z InvocationID=a6104310-f554-4ea5-ad93-fc47380207c8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:33.243Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:31.435Z InvocationID=a6104310-f554-4ea5-ad93-fc47380207c8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:34.128Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:31.435Z InvocationID=a6104310-f554-4ea5-ad93-fc47380207c8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Vdu 2022-04-26T06:24:34.546Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06f60fdd-f222-4fd1-89a8-4f30ac436f3b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:24:34.546Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06f60fdd-f222-4fd1-89a8-4f30ac436f3b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4e6b8565 2022-04-26T06:24:34.546Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06f60fdd-f222-4fd1-89a8-4f30ac436f3b PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:24:34.546Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06f60fdd-f222-4fd1-89a8-4f30ac436f3b PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:24:34.830Z [qtp2085002312-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "03ebdaa5-c215-49a5-8bb9-f1095f35e9a6" INVARIANT_UUID = "a9cb86e8-8b48-415e-9bc6-49a9b41af340" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VDU" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=03ebdaa5-c215-49a5-8bb9-f1095f35e9a6 EntryTimestamp=2022-04-26T06:24:31.435Z InvocationID=a6104310-f554-4ea5-ad93-fc47380207c8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:24:34.832Z [qtp2085002312-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "03ebdaa5-c215-49a5-8bb9-f1095f35e9a6" INVARIANT_UUID = "a9cb86e8-8b48-415e-9bc6-49a9b41af340" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VDU" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=03ebdaa5-c215-49a5-8bb9-f1095f35e9a6 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:31.435Z InvocationID=a6104310-f554-4ea5-ad93-fc47380207c8 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:24:35.053Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "03ebdaa5-c215-49a5-8bb9-f1095f35e9a6" INVARIANT_UUID = "a9cb86e8-8b48-415e-9bc6-49a9b41af340" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VDU" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=03ebdaa5-c215-49a5-8bb9-f1095f35e9a6 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:31.435Z InvocationID=a6104310-f554-4ea5-ad93-fc47380207c8 ErrorCode=500 lock resource with id d17b4606-9d79-4e50-97e9-7c3cb7c8e7ab 2022-04-26T06:24:35.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:24:35.379Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:24:35.561Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=06f60fdd-f222-4fd1-89a8-4f30ac436f3b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:24:35.561Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=06f60fdd-f222-4fd1-89a8-4f30ac436f3b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:24:36.838Z [qtp2085002312-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "03ebdaa5-c215-49a5-8bb9-f1095f35e9a6" INVARIANT_UUID = "a9cb86e8-8b48-415e-9bc6-49a9b41af340" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "d17b4606-9d79-4e50-97e9-7c3cb7c8e7ab.assettoscatemplate" CURR_ARTIFACT_UUID = "d17b4606-9d79-4e50-97e9-7c3cb7c8e7ab.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vdu-template.yml,0,d17b4606-9d79-4e50-97e9-7c3cb7c8e7ab.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8096d0ca-b756-4c9a-95a7-082a2eefab4c TargetServiceName=null EntryTimestamp=2022-04-26T06:24:31.435Z InvocationID=a6104310-f554-4ea5-ad93-fc47380207c8 ErrorCode=500 Starting to create Csar for component VDU 2022-04-26T06:24:37.147Z [qtp2085002312-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "03ebdaa5-c215-49a5-8bb9-f1095f35e9a6" INVARIANT_UUID = "a9cb86e8-8b48-415e-9bc6-49a9b41af340" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "d17b4606-9d79-4e50-97e9-7c3cb7c8e7ab.assettoscatemplate" CURR_ARTIFACT_UUID = "d17b4606-9d79-4e50-97e9-7c3cb7c8e7ab.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vdu-template.yml,0,d17b4606-9d79-4e50-97e9-7c3cb7c8e7ab.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8096d0ca-b756-4c9a-95a7-082a2eefab4c TargetServiceName=null EntryTimestamp=2022-04-26T06:24:31.435Z InvocationID=a6104310-f554-4ea5-ad93-fc47380207c8 ErrorCode=500 Ended create Csar for component VDU 2022-04-26T06:24:37.148Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "03ebdaa5-c215-49a5-8bb9-f1095f35e9a6" INVARIANT_UUID = "a9cb86e8-8b48-415e-9bc6-49a9b41af340" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "d17b4606-9d79-4e50-97e9-7c3cb7c8e7ab.assettoscatemplate" CURR_ARTIFACT_UUID = "d17b4606-9d79-4e50-97e9-7c3cb7c8e7ab.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vdu-template.yml,0,d17b4606-9d79-4e50-97e9-7c3cb7c8e7ab.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8096d0ca-b756-4c9a-95a7-082a2eefab4c TargetServiceName=null EntryTimestamp=2022-04-26T06:24:31.435Z InvocationID=a6104310-f554-4ea5-ad93-fc47380207c8 ErrorCode=500 lock resource with id d17b4606-9d79-4e50-97e9-7c3cb7c8e7ab 2022-04-26T06:24:37.942Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47854a5a-def2-486a-a00a-5dd542548cbc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:24:37.942Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47854a5a-def2-486a-a00a-5dd542548cbc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@24eaf227 2022-04-26T06:24:37.942Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47854a5a-def2-486a-a00a-5dd542548cbc PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:24:37.942Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47854a5a-def2-486a-a00a-5dd542548cbc PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:24:38.256Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "03ebdaa5-c215-49a5-8bb9-f1095f35e9a6" INVARIANT_UUID = "a9cb86e8-8b48-415e-9bc6-49a9b41af340" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "d17b4606-9d79-4e50-97e9-7c3cb7c8e7ab.assettoscacsar" CURR_ARTIFACT_UUID = "d17b4606-9d79-4e50-97e9-7c3cb7c8e7ab.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Vdu-csar.csar,0,d17b4606-9d79-4e50-97e9-7c3cb7c8e7ab.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=6821 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9ff7372d-71be-4b83-b2bb-4a968730d85b TargetServiceName=null EntryTimestamp=2022-04-26T06:24:31.435Z InvocationID=a6104310-f554-4ea5-ad93-fc47380207c8 ErrorCode=500 LogTimestamp=2022-04-26T06:24:38.256134Z ResponseDesc=Created Exiting. 2022-04-26T06:24:38.257Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:24:38.333Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=e0a2a623-2864-4e76-bae5-d9e579192e0c InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:38.332717Z InvocationID=cd8d045a-7a5b-4a2a-a936-98ac534cdad6 LogTimestamp=2022-04-26T06:24:38.332783Z Entering 2022-04-26T06:24:38.333Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=e0a2a623-2864-4e76-bae5-d9e579192e0c InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:38.332717Z InvocationID=cd8d045a-7a5b-4a2a-a936-98ac534cdad6 LogTimestamp=2022-04-26T06:24:38.332783Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:24:38.993Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=47854a5a-def2-486a-a00a-5dd542548cbc ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:24:38.994Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=47854a5a-def2-486a-a00a-5dd542548cbc ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:24:39.363Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:38.333Z InvocationID=530545b5-932f-4955-9594-c3d0fc362b17 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:39.363Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:38.333Z InvocationID=530545b5-932f-4955-9594-c3d0fc362b17 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:39.366Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:38.333Z InvocationID=530545b5-932f-4955-9594-c3d0fc362b17 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:39.366Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:38.333Z InvocationID=530545b5-932f-4955-9594-c3d0fc362b17 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:39.942Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:38.333Z InvocationID=530545b5-932f-4955-9594-c3d0fc362b17 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name VduCompute 2022-04-26T06:24:40.348Z [qtp2085002312-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "286e8f95-e9d3-4fb8-807b-0bfe9e693cc6" INVARIANT_UUID = "434d09ad-ba73-49db-93e0-4fd09ccbf832" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.Compute" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=286e8f95-e9d3-4fb8-807b-0bfe9e693cc6 EntryTimestamp=2022-04-26T06:24:38.333Z InvocationID=530545b5-932f-4955-9594-c3d0fc362b17 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:24:40.348Z [qtp2085002312-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "286e8f95-e9d3-4fb8-807b-0bfe9e693cc6" INVARIANT_UUID = "434d09ad-ba73-49db-93e0-4fd09ccbf832" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.Compute" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=286e8f95-e9d3-4fb8-807b-0bfe9e693cc6 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:38.333Z InvocationID=530545b5-932f-4955-9594-c3d0fc362b17 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:24:40.352Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0175b554-97b7-4333-86df-c71342c40f08 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:24:40.352Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0175b554-97b7-4333-86df-c71342c40f08 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f68b45b 2022-04-26T06:24:40.352Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0175b554-97b7-4333-86df-c71342c40f08 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:24:40.352Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0175b554-97b7-4333-86df-c71342c40f08 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:24:40.467Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "286e8f95-e9d3-4fb8-807b-0bfe9e693cc6" INVARIANT_UUID = "434d09ad-ba73-49db-93e0-4fd09ccbf832" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.Compute" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=286e8f95-e9d3-4fb8-807b-0bfe9e693cc6 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:38.333Z InvocationID=530545b5-932f-4955-9594-c3d0fc362b17 ErrorCode=500 lock resource with id c439ea4c-eb18-4f10-98c1-37336155dd48 2022-04-26T06:24:41.073Z [qtp2085002312-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "286e8f95-e9d3-4fb8-807b-0bfe9e693cc6" INVARIANT_UUID = "434d09ad-ba73-49db-93e0-4fd09ccbf832" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c439ea4c-eb18-4f10-98c1-37336155dd48.assettoscatemplate" CURR_ARTIFACT_UUID = "c439ea4c-eb18-4f10-98c1-37336155dd48.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCompute-template.yml,0,c439ea4c-eb18-4f10-98c1-37336155dd48.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=62ce910d-5b28-4f30-ba35-9c4159ced199 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:38.333Z InvocationID=530545b5-932f-4955-9594-c3d0fc362b17 ErrorCode=500 Starting to create Csar for component VDU Compute 2022-04-26T06:24:41.390Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=0175b554-97b7-4333-86df-c71342c40f08 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:24:41.390Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=0175b554-97b7-4333-86df-c71342c40f08 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:24:41.392Z [qtp2085002312-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "286e8f95-e9d3-4fb8-807b-0bfe9e693cc6" INVARIANT_UUID = "434d09ad-ba73-49db-93e0-4fd09ccbf832" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c439ea4c-eb18-4f10-98c1-37336155dd48.assettoscatemplate" CURR_ARTIFACT_UUID = "c439ea4c-eb18-4f10-98c1-37336155dd48.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCompute-template.yml,0,c439ea4c-eb18-4f10-98c1-37336155dd48.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=62ce910d-5b28-4f30-ba35-9c4159ced199 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:38.333Z InvocationID=530545b5-932f-4955-9594-c3d0fc362b17 ErrorCode=500 Ended create Csar for component VDU Compute 2022-04-26T06:24:41.393Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "286e8f95-e9d3-4fb8-807b-0bfe9e693cc6" INVARIANT_UUID = "434d09ad-ba73-49db-93e0-4fd09ccbf832" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c439ea4c-eb18-4f10-98c1-37336155dd48.assettoscatemplate" CURR_ARTIFACT_UUID = "c439ea4c-eb18-4f10-98c1-37336155dd48.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCompute-template.yml,0,c439ea4c-eb18-4f10-98c1-37336155dd48.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=62ce910d-5b28-4f30-ba35-9c4159ced199 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:38.333Z InvocationID=530545b5-932f-4955-9594-c3d0fc362b17 ErrorCode=500 lock resource with id c439ea4c-eb18-4f10-98c1-37336155dd48 2022-04-26T06:24:41.950Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "286e8f95-e9d3-4fb8-807b-0bfe9e693cc6" INVARIANT_UUID = "434d09ad-ba73-49db-93e0-4fd09ccbf832" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c439ea4c-eb18-4f10-98c1-37336155dd48.assettoscacsar" CURR_ARTIFACT_UUID = "c439ea4c-eb18-4f10-98c1-37336155dd48.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduCompute-csar.csar,0,c439ea4c-eb18-4f10-98c1-37336155dd48.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=3617 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=eb1efa2d-8d88-4940-ae43-a91417d3e4ed TargetServiceName=null EntryTimestamp=2022-04-26T06:24:38.333Z InvocationID=530545b5-932f-4955-9594-c3d0fc362b17 ErrorCode=500 LogTimestamp=2022-04-26T06:24:41.950316Z ResponseDesc=Created Exiting. 2022-04-26T06:24:41.951Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:24:41.967Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=7921c995-eb22-43ea-a192-04c2916215c5 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:41.967403Z InvocationID=e7474dcc-076c-487b-9d19-1c4b93702cc4 LogTimestamp=2022-04-26T06:24:41.967500Z Entering 2022-04-26T06:24:41.968Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=7921c995-eb22-43ea-a192-04c2916215c5 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:41.967403Z InvocationID=e7474dcc-076c-487b-9d19-1c4b93702cc4 LogTimestamp=2022-04-26T06:24:41.967500Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:24:42.168Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:41.968Z InvocationID=39e329c1-9225-48e3-b2e1-2aa5a6b31c59 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:42.168Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:41.968Z InvocationID=39e329c1-9225-48e3-b2e1-2aa5a6b31c59 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:42.169Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:41.968Z InvocationID=39e329c1-9225-48e3-b2e1-2aa5a6b31c59 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:42.169Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:41.968Z InvocationID=39e329c1-9225-48e3-b2e1-2aa5a6b31c59 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:42.169Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:41.968Z InvocationID=39e329c1-9225-48e3-b2e1-2aa5a6b31c59 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Cp 2022-04-26T06:24:42.218Z [qtp2085002312-20] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Cp" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "faf4c93b-991a-47af-b72e-aa21d55417d7" INVARIANT_UUID = "6a5b379e-1d46-455d-8580-1c2e2e676d3d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Cp" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=faf4c93b-991a-47af-b72e-aa21d55417d7 EntryTimestamp=2022-04-26T06:24:41.968Z InvocationID=39e329c1-9225-48e3-b2e1-2aa5a6b31c59 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:24:42.219Z [qtp2085002312-20] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Cp" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "faf4c93b-991a-47af-b72e-aa21d55417d7" INVARIANT_UUID = "6a5b379e-1d46-455d-8580-1c2e2e676d3d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Cp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=faf4c93b-991a-47af-b72e-aa21d55417d7 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:41.968Z InvocationID=39e329c1-9225-48e3-b2e1-2aa5a6b31c59 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:24:42.271Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Cp" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "faf4c93b-991a-47af-b72e-aa21d55417d7" INVARIANT_UUID = "6a5b379e-1d46-455d-8580-1c2e2e676d3d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Cp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=faf4c93b-991a-47af-b72e-aa21d55417d7 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:41.968Z InvocationID=39e329c1-9225-48e3-b2e1-2aa5a6b31c59 ErrorCode=500 lock resource with id 8184af52-2bd0-4e57-a785-4f5470b96afd 2022-04-26T06:24:42.789Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e09e031-9a60-4d11-bfca-3c0b990519fa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:24:42.790Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e09e031-9a60-4d11-bfca-3c0b990519fa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@618d8c62 2022-04-26T06:24:42.790Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e09e031-9a60-4d11-bfca-3c0b990519fa PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:24:42.790Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e09e031-9a60-4d11-bfca-3c0b990519fa PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:24:42.951Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Cp" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "faf4c93b-991a-47af-b72e-aa21d55417d7" INVARIANT_UUID = "6a5b379e-1d46-455d-8580-1c2e2e676d3d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8184af52-2bd0-4e57-a785-4f5470b96afd.assettoscatemplate" CURR_ARTIFACT_UUID = "8184af52-2bd0-4e57-a785-4f5470b96afd.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Cp-template.yml,0,8184af52-2bd0-4e57-a785-4f5470b96afd.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a4b014fa-6d27-4792-8a44-3d5da2feb5e8 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:41.968Z InvocationID=39e329c1-9225-48e3-b2e1-2aa5a6b31c59 ErrorCode=500 Starting to create Csar for component Cp 2022-04-26T06:24:43.128Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Cp" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "faf4c93b-991a-47af-b72e-aa21d55417d7" INVARIANT_UUID = "6a5b379e-1d46-455d-8580-1c2e2e676d3d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8184af52-2bd0-4e57-a785-4f5470b96afd.assettoscatemplate" CURR_ARTIFACT_UUID = "8184af52-2bd0-4e57-a785-4f5470b96afd.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Cp-template.yml,0,8184af52-2bd0-4e57-a785-4f5470b96afd.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a4b014fa-6d27-4792-8a44-3d5da2feb5e8 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:41.968Z InvocationID=39e329c1-9225-48e3-b2e1-2aa5a6b31c59 ErrorCode=500 Ended create Csar for component Cp 2022-04-26T06:24:43.128Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Cp" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "faf4c93b-991a-47af-b72e-aa21d55417d7" INVARIANT_UUID = "6a5b379e-1d46-455d-8580-1c2e2e676d3d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8184af52-2bd0-4e57-a785-4f5470b96afd.assettoscatemplate" CURR_ARTIFACT_UUID = "8184af52-2bd0-4e57-a785-4f5470b96afd.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Cp-template.yml,0,8184af52-2bd0-4e57-a785-4f5470b96afd.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a4b014fa-6d27-4792-8a44-3d5da2feb5e8 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:41.968Z InvocationID=39e329c1-9225-48e3-b2e1-2aa5a6b31c59 ErrorCode=500 lock resource with id 8184af52-2bd0-4e57-a785-4f5470b96afd 2022-04-26T06:24:43.676Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Cp" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "faf4c93b-991a-47af-b72e-aa21d55417d7" INVARIANT_UUID = "6a5b379e-1d46-455d-8580-1c2e2e676d3d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8184af52-2bd0-4e57-a785-4f5470b96afd.assettoscacsar" CURR_ARTIFACT_UUID = "8184af52-2bd0-4e57-a785-4f5470b96afd.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Cp-csar.csar,0,8184af52-2bd0-4e57-a785-4f5470b96afd.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1708 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f4781bbf-b50b-42da-89db-f343d9acb646 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:41.968Z InvocationID=39e329c1-9225-48e3-b2e1-2aa5a6b31c59 ErrorCode=500 LogTimestamp=2022-04-26T06:24:43.676060Z ResponseDesc=Created Exiting. 2022-04-26T06:24:43.676Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:24:43.690Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=9a51bb76-b6b5-45d6-aee8-3ee6ba3d468b InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:43.690529Z InvocationID=d4ca6e29-6740-4cf3-97d7-1342733a654e LogTimestamp=2022-04-26T06:24:43.690595Z Entering 2022-04-26T06:24:43.691Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=9a51bb76-b6b5-45d6-aee8-3ee6ba3d468b InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:43.690529Z InvocationID=d4ca6e29-6740-4cf3-97d7-1342733a654e LogTimestamp=2022-04-26T06:24:43.690595Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:24:43.791Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:43.691Z InvocationID=874104ac-18fb-4e52-8130-f04fe5c4724a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:43.791Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:43.691Z InvocationID=874104ac-18fb-4e52-8130-f04fe5c4724a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:43.791Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:43.691Z InvocationID=874104ac-18fb-4e52-8130-f04fe5c4724a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:43.791Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:43.691Z InvocationID=874104ac-18fb-4e52-8130-f04fe5c4724a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:43.798Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:43.691Z InvocationID=874104ac-18fb-4e52-8130-f04fe5c4724a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name VduVirtualstorage 2022-04-26T06:24:43.822Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=5e09e031-9a60-4d11-bfca-3c0b990519fa ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:24:43.822Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=5e09e031-9a60-4d11-bfca-3c0b990519fa ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:24:43.848Z [qtp2085002312-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "159e6b08-840b-4219-ad3d-a6b9dc95a5f9" INVARIANT_UUID = "e8a3320e-c161-46f5-bafd-e070d7fde9c5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualStorage" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=159e6b08-840b-4219-ad3d-a6b9dc95a5f9 EntryTimestamp=2022-04-26T06:24:43.691Z InvocationID=874104ac-18fb-4e52-8130-f04fe5c4724a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:24:43.849Z [qtp2085002312-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "159e6b08-840b-4219-ad3d-a6b9dc95a5f9" INVARIANT_UUID = "e8a3320e-c161-46f5-bafd-e070d7fde9c5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=159e6b08-840b-4219-ad3d-a6b9dc95a5f9 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:43.691Z InvocationID=874104ac-18fb-4e52-8130-f04fe5c4724a ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:24:43.892Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "159e6b08-840b-4219-ad3d-a6b9dc95a5f9" INVARIANT_UUID = "e8a3320e-c161-46f5-bafd-e070d7fde9c5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=159e6b08-840b-4219-ad3d-a6b9dc95a5f9 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:43.691Z InvocationID=874104ac-18fb-4e52-8130-f04fe5c4724a ErrorCode=500 lock resource with id a2847028-9440-4c16-82f6-5c201a240423 2022-04-26T06:24:44.489Z [qtp2085002312-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "159e6b08-840b-4219-ad3d-a6b9dc95a5f9" INVARIANT_UUID = "e8a3320e-c161-46f5-bafd-e070d7fde9c5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a2847028-9440-4c16-82f6-5c201a240423.assettoscatemplate" CURR_ARTIFACT_UUID = "a2847028-9440-4c16-82f6-5c201a240423.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualstorage-template.yml,0,a2847028-9440-4c16-82f6-5c201a240423.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d414e009-5ddc-4b8c-b871-3ce5d34d0d28 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:43.691Z InvocationID=874104ac-18fb-4e52-8130-f04fe5c4724a ErrorCode=500 Starting to create Csar for component VDU VirtualStorage 2022-04-26T06:24:44.639Z [qtp2085002312-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "159e6b08-840b-4219-ad3d-a6b9dc95a5f9" INVARIANT_UUID = "e8a3320e-c161-46f5-bafd-e070d7fde9c5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a2847028-9440-4c16-82f6-5c201a240423.assettoscatemplate" CURR_ARTIFACT_UUID = "a2847028-9440-4c16-82f6-5c201a240423.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualstorage-template.yml,0,a2847028-9440-4c16-82f6-5c201a240423.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d414e009-5ddc-4b8c-b871-3ce5d34d0d28 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:43.691Z InvocationID=874104ac-18fb-4e52-8130-f04fe5c4724a ErrorCode=500 Ended create Csar for component VDU VirtualStorage 2022-04-26T06:24:44.639Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "159e6b08-840b-4219-ad3d-a6b9dc95a5f9" INVARIANT_UUID = "e8a3320e-c161-46f5-bafd-e070d7fde9c5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a2847028-9440-4c16-82f6-5c201a240423.assettoscatemplate" CURR_ARTIFACT_UUID = "a2847028-9440-4c16-82f6-5c201a240423.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualstorage-template.yml,0,a2847028-9440-4c16-82f6-5c201a240423.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d414e009-5ddc-4b8c-b871-3ce5d34d0d28 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:43.691Z InvocationID=874104ac-18fb-4e52-8130-f04fe5c4724a ErrorCode=500 lock resource with id a2847028-9440-4c16-82f6-5c201a240423 2022-04-26T06:24:45.210Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "159e6b08-840b-4219-ad3d-a6b9dc95a5f9" INVARIANT_UUID = "e8a3320e-c161-46f5-bafd-e070d7fde9c5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a2847028-9440-4c16-82f6-5c201a240423.assettoscacsar" CURR_ARTIFACT_UUID = "a2847028-9440-4c16-82f6-5c201a240423.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduVirtualstorage-csar.csar,0,a2847028-9440-4c16-82f6-5c201a240423.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1519 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b8ba6926-e34b-4c24-9a90-531533066552 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:43.691Z InvocationID=874104ac-18fb-4e52-8130-f04fe5c4724a ErrorCode=500 LogTimestamp=2022-04-26T06:24:45.210028Z ResponseDesc=Created Exiting. 2022-04-26T06:24:45.211Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:24:45.224Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=df7e86c4-5928-4506-892a-1023b413d656 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:45.224460Z InvocationID=aea3e34e-2b2a-4bfe-a338-2af82c01a5fe LogTimestamp=2022-04-26T06:24:45.224520Z Entering 2022-04-26T06:24:45.225Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=df7e86c4-5928-4506-892a-1023b413d656 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:45.224460Z InvocationID=aea3e34e-2b2a-4bfe-a338-2af82c01a5fe LogTimestamp=2022-04-26T06:24:45.224520Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:24:45.349Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:45.225Z InvocationID=2c5020b1-af9a-464a-b118-6916e5e649bf PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:45.349Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:45.225Z InvocationID=2c5020b1-af9a-464a-b118-6916e5e649bf PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:45.349Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:45.225Z InvocationID=2c5020b1-af9a-464a-b118-6916e5e649bf PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:45.349Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:45.225Z InvocationID=2c5020b1-af9a-464a-b118-6916e5e649bf PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:45.354Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:45.225Z InvocationID=2c5020b1-af9a-464a-b118-6916e5e649bf PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name VduVirtualblockstorage 2022-04-26T06:24:45.403Z [qtp2085002312-20] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "dbf0616e-b7ea-4117-8b1d-ef4d388422cb" INVARIANT_UUID = "b006ae6a-aad8-4be2-9315-2c995dbd801e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualBlockStorage" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=dbf0616e-b7ea-4117-8b1d-ef4d388422cb EntryTimestamp=2022-04-26T06:24:45.225Z InvocationID=2c5020b1-af9a-464a-b118-6916e5e649bf PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:24:45.404Z [qtp2085002312-20] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "dbf0616e-b7ea-4117-8b1d-ef4d388422cb" INVARIANT_UUID = "b006ae6a-aad8-4be2-9315-2c995dbd801e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualBlockStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=dbf0616e-b7ea-4117-8b1d-ef4d388422cb TargetServiceName=null EntryTimestamp=2022-04-26T06:24:45.225Z InvocationID=2c5020b1-af9a-464a-b118-6916e5e649bf ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:24:45.434Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "dbf0616e-b7ea-4117-8b1d-ef4d388422cb" INVARIANT_UUID = "b006ae6a-aad8-4be2-9315-2c995dbd801e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualBlockStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=dbf0616e-b7ea-4117-8b1d-ef4d388422cb TargetServiceName=null EntryTimestamp=2022-04-26T06:24:45.225Z InvocationID=2c5020b1-af9a-464a-b118-6916e5e649bf ErrorCode=500 lock resource with id 44465c5d-d82b-406e-9f0c-9731e383f4d6 2022-04-26T06:24:45.795Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f6c77de-ea67-4a98-8276-4d8aaf862a63 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:24:45.795Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f6c77de-ea67-4a98-8276-4d8aaf862a63 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@648982eb 2022-04-26T06:24:45.795Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f6c77de-ea67-4a98-8276-4d8aaf862a63 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:24:45.795Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f6c77de-ea67-4a98-8276-4d8aaf862a63 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:24:46.047Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "dbf0616e-b7ea-4117-8b1d-ef4d388422cb" INVARIANT_UUID = "b006ae6a-aad8-4be2-9315-2c995dbd801e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "44465c5d-d82b-406e-9f0c-9731e383f4d6.assettoscatemplate" CURR_ARTIFACT_UUID = "44465c5d-d82b-406e-9f0c-9731e383f4d6.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualblockstorage-template.yml,0,44465c5d-d82b-406e-9f0c-9731e383f4d6.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=87539a54-2cb2-41e4-9ceb-b7b45752266e TargetServiceName=null EntryTimestamp=2022-04-26T06:24:45.225Z InvocationID=2c5020b1-af9a-464a-b118-6916e5e649bf ErrorCode=500 Starting to create Csar for component VDU VirtualBlockStorage 2022-04-26T06:24:46.202Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "dbf0616e-b7ea-4117-8b1d-ef4d388422cb" INVARIANT_UUID = "b006ae6a-aad8-4be2-9315-2c995dbd801e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "44465c5d-d82b-406e-9f0c-9731e383f4d6.assettoscatemplate" CURR_ARTIFACT_UUID = "44465c5d-d82b-406e-9f0c-9731e383f4d6.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualblockstorage-template.yml,0,44465c5d-d82b-406e-9f0c-9731e383f4d6.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=87539a54-2cb2-41e4-9ceb-b7b45752266e TargetServiceName=null EntryTimestamp=2022-04-26T06:24:45.225Z InvocationID=2c5020b1-af9a-464a-b118-6916e5e649bf ErrorCode=500 Ended create Csar for component VDU VirtualBlockStorage 2022-04-26T06:24:46.203Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "dbf0616e-b7ea-4117-8b1d-ef4d388422cb" INVARIANT_UUID = "b006ae6a-aad8-4be2-9315-2c995dbd801e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "44465c5d-d82b-406e-9f0c-9731e383f4d6.assettoscatemplate" CURR_ARTIFACT_UUID = "44465c5d-d82b-406e-9f0c-9731e383f4d6.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualblockstorage-template.yml,0,44465c5d-d82b-406e-9f0c-9731e383f4d6.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=87539a54-2cb2-41e4-9ceb-b7b45752266e TargetServiceName=null EntryTimestamp=2022-04-26T06:24:45.225Z InvocationID=2c5020b1-af9a-464a-b118-6916e5e649bf ErrorCode=500 lock resource with id 44465c5d-d82b-406e-9f0c-9731e383f4d6 2022-04-26T06:24:46.769Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "dbf0616e-b7ea-4117-8b1d-ef4d388422cb" INVARIANT_UUID = "b006ae6a-aad8-4be2-9315-2c995dbd801e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "44465c5d-d82b-406e-9f0c-9731e383f4d6.assettoscacsar" CURR_ARTIFACT_UUID = "44465c5d-d82b-406e-9f0c-9731e383f4d6.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduVirtualblockstorage-csar.csar,0,44465c5d-d82b-406e-9f0c-9731e383f4d6.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1544 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8be15e9c-d2cf-4ae2-b65e-9bfb574a590f TargetServiceName=null EntryTimestamp=2022-04-26T06:24:45.225Z InvocationID=2c5020b1-af9a-464a-b118-6916e5e649bf ErrorCode=500 LogTimestamp=2022-04-26T06:24:46.769333Z ResponseDesc=Created Exiting. 2022-04-26T06:24:46.770Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:24:46.783Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=4b71929e-81bc-4457-a232-daf841def672 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:46.782762Z InvocationID=e6c18d67-c767-4256-8102-58cc470c9e32 LogTimestamp=2022-04-26T06:24:46.782817Z Entering 2022-04-26T06:24:46.783Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=4b71929e-81bc-4457-a232-daf841def672 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:46.782762Z InvocationID=e6c18d67-c767-4256-8102-58cc470c9e32 LogTimestamp=2022-04-26T06:24:46.782817Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:24:46.825Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=6f6c77de-ea67-4a98-8276-4d8aaf862a63 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:24:46.825Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=6f6c77de-ea67-4a98-8276-4d8aaf862a63 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:24:46.874Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:46.783Z InvocationID=614c0196-00d3-4d72-84ed-21b1f9376811 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:46.875Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:46.783Z InvocationID=614c0196-00d3-4d72-84ed-21b1f9376811 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:46.875Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:46.783Z InvocationID=614c0196-00d3-4d72-84ed-21b1f9376811 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:46.875Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:46.783Z InvocationID=614c0196-00d3-4d72-84ed-21b1f9376811 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:46.883Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:46.783Z InvocationID=614c0196-00d3-4d72-84ed-21b1f9376811 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name VduVirtualfilestorage 2022-04-26T06:24:46.964Z [qtp2085002312-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "bfdd7046-6db7-44fd-bb9b-a9f4605073c6" INVARIANT_UUID = "125d1e27-af6c-4dd1-af02-c72d377c47c5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualFileStorage" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bfdd7046-6db7-44fd-bb9b-a9f4605073c6 EntryTimestamp=2022-04-26T06:24:46.783Z InvocationID=614c0196-00d3-4d72-84ed-21b1f9376811 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:24:46.965Z [qtp2085002312-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "bfdd7046-6db7-44fd-bb9b-a9f4605073c6" INVARIANT_UUID = "125d1e27-af6c-4dd1-af02-c72d377c47c5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualFileStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bfdd7046-6db7-44fd-bb9b-a9f4605073c6 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:46.783Z InvocationID=614c0196-00d3-4d72-84ed-21b1f9376811 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:24:46.998Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "bfdd7046-6db7-44fd-bb9b-a9f4605073c6" INVARIANT_UUID = "125d1e27-af6c-4dd1-af02-c72d377c47c5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualFileStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bfdd7046-6db7-44fd-bb9b-a9f4605073c6 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:46.783Z InvocationID=614c0196-00d3-4d72-84ed-21b1f9376811 ErrorCode=500 lock resource with id cf3efd4e-334d-409f-8a1d-f7bbbf2479aa 2022-04-26T06:24:47.577Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "bfdd7046-6db7-44fd-bb9b-a9f4605073c6" INVARIANT_UUID = "125d1e27-af6c-4dd1-af02-c72d377c47c5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "cf3efd4e-334d-409f-8a1d-f7bbbf2479aa.assettoscatemplate" CURR_ARTIFACT_UUID = "cf3efd4e-334d-409f-8a1d-f7bbbf2479aa.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualfilestorage-template.yml,0,cf3efd4e-334d-409f-8a1d-f7bbbf2479aa.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e4a20ebb-0e3e-4af4-85f7-c977b034408c TargetServiceName=null EntryTimestamp=2022-04-26T06:24:46.783Z InvocationID=614c0196-00d3-4d72-84ed-21b1f9376811 ErrorCode=500 Starting to create Csar for component VDU VirtualFileStorage 2022-04-26T06:24:47.754Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "bfdd7046-6db7-44fd-bb9b-a9f4605073c6" INVARIANT_UUID = "125d1e27-af6c-4dd1-af02-c72d377c47c5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "cf3efd4e-334d-409f-8a1d-f7bbbf2479aa.assettoscatemplate" CURR_ARTIFACT_UUID = "cf3efd4e-334d-409f-8a1d-f7bbbf2479aa.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualfilestorage-template.yml,0,cf3efd4e-334d-409f-8a1d-f7bbbf2479aa.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e4a20ebb-0e3e-4af4-85f7-c977b034408c TargetServiceName=null EntryTimestamp=2022-04-26T06:24:46.783Z InvocationID=614c0196-00d3-4d72-84ed-21b1f9376811 ErrorCode=500 Ended create Csar for component VDU VirtualFileStorage 2022-04-26T06:24:47.755Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "bfdd7046-6db7-44fd-bb9b-a9f4605073c6" INVARIANT_UUID = "125d1e27-af6c-4dd1-af02-c72d377c47c5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "cf3efd4e-334d-409f-8a1d-f7bbbf2479aa.assettoscatemplate" CURR_ARTIFACT_UUID = "cf3efd4e-334d-409f-8a1d-f7bbbf2479aa.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualfilestorage-template.yml,0,cf3efd4e-334d-409f-8a1d-f7bbbf2479aa.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e4a20ebb-0e3e-4af4-85f7-c977b034408c TargetServiceName=null EntryTimestamp=2022-04-26T06:24:46.783Z InvocationID=614c0196-00d3-4d72-84ed-21b1f9376811 ErrorCode=500 lock resource with id cf3efd4e-334d-409f-8a1d-f7bbbf2479aa 2022-04-26T06:24:48.305Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "bfdd7046-6db7-44fd-bb9b-a9f4605073c6" INVARIANT_UUID = "125d1e27-af6c-4dd1-af02-c72d377c47c5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "cf3efd4e-334d-409f-8a1d-f7bbbf2479aa.assettoscacsar" CURR_ARTIFACT_UUID = "cf3efd4e-334d-409f-8a1d-f7bbbf2479aa.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduVirtualfilestorage-csar.csar,0,cf3efd4e-334d-409f-8a1d-f7bbbf2479aa.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1522 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=eda32671-d86d-4337-abcb-fccc6db5e7b2 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:46.783Z InvocationID=614c0196-00d3-4d72-84ed-21b1f9376811 ErrorCode=500 LogTimestamp=2022-04-26T06:24:48.305408Z ResponseDesc=Created Exiting. 2022-04-26T06:24:48.310Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:24:48.323Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=7fe9b8fa-3890-4c84-a582-86cd1937706a InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:48.323136Z InvocationID=d19cce69-5c37-4580-b6a3-6d44ee5bd0f9 LogTimestamp=2022-04-26T06:24:48.323216Z Entering 2022-04-26T06:24:48.323Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=7fe9b8fa-3890-4c84-a582-86cd1937706a InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:48.323136Z InvocationID=d19cce69-5c37-4580-b6a3-6d44ee5bd0f9 LogTimestamp=2022-04-26T06:24:48.323216Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:24:48.415Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:48.324Z InvocationID=ec48b520-a6e1-4140-a76f-043bc512a1ff PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:48.416Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:48.324Z InvocationID=ec48b520-a6e1-4140-a76f-043bc512a1ff PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:48.416Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:48.324Z InvocationID=ec48b520-a6e1-4140-a76f-043bc512a1ff PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:48.416Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:48.324Z InvocationID=ec48b520-a6e1-4140-a76f-043bc512a1ff PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:48.420Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:48.324Z InvocationID=ec48b520-a6e1-4140-a76f-043bc512a1ff PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name VduVirtualobjectstorage 2022-04-26T06:24:48.457Z [qtp2085002312-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "7aaf628f-8a3e-47d8-9aa0-f4a0050c7ee6" INVARIANT_UUID = "edc6bf35-e9c8-497d-b8c6-55dde16f9f26" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualObjectStorage" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7aaf628f-8a3e-47d8-9aa0-f4a0050c7ee6 EntryTimestamp=2022-04-26T06:24:48.324Z InvocationID=ec48b520-a6e1-4140-a76f-043bc512a1ff PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:24:48.458Z [qtp2085002312-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "7aaf628f-8a3e-47d8-9aa0-f4a0050c7ee6" INVARIANT_UUID = "edc6bf35-e9c8-497d-b8c6-55dde16f9f26" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualObjectStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7aaf628f-8a3e-47d8-9aa0-f4a0050c7ee6 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:48.324Z InvocationID=ec48b520-a6e1-4140-a76f-043bc512a1ff ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:24:48.496Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "7aaf628f-8a3e-47d8-9aa0-f4a0050c7ee6" INVARIANT_UUID = "edc6bf35-e9c8-497d-b8c6-55dde16f9f26" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualObjectStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7aaf628f-8a3e-47d8-9aa0-f4a0050c7ee6 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:48.324Z InvocationID=ec48b520-a6e1-4140-a76f-043bc512a1ff ErrorCode=500 lock resource with id 09c69ff1-e322-468c-b980-b0a792d6a695 2022-04-26T06:24:48.798Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c1a14c4-1ee9-479b-9dc1-c48bc22ff194 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:24:48.798Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c1a14c4-1ee9-479b-9dc1-c48bc22ff194 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@522a4be5 2022-04-26T06:24:48.798Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c1a14c4-1ee9-479b-9dc1-c48bc22ff194 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:24:48.798Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c1a14c4-1ee9-479b-9dc1-c48bc22ff194 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:24:49.098Z [qtp2085002312-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "7aaf628f-8a3e-47d8-9aa0-f4a0050c7ee6" INVARIANT_UUID = "edc6bf35-e9c8-497d-b8c6-55dde16f9f26" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "09c69ff1-e322-468c-b980-b0a792d6a695.assettoscatemplate" CURR_ARTIFACT_UUID = "09c69ff1-e322-468c-b980-b0a792d6a695.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualobjectstorage-template.yml,0,09c69ff1-e322-468c-b980-b0a792d6a695.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=476ed07d-a35d-43f6-9ac3-b9aff31b20f1 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:48.324Z InvocationID=ec48b520-a6e1-4140-a76f-043bc512a1ff ErrorCode=500 Starting to create Csar for component VDU VirtualObjectStorage 2022-04-26T06:24:49.290Z [qtp2085002312-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "7aaf628f-8a3e-47d8-9aa0-f4a0050c7ee6" INVARIANT_UUID = "edc6bf35-e9c8-497d-b8c6-55dde16f9f26" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "09c69ff1-e322-468c-b980-b0a792d6a695.assettoscatemplate" CURR_ARTIFACT_UUID = "09c69ff1-e322-468c-b980-b0a792d6a695.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualobjectstorage-template.yml,0,09c69ff1-e322-468c-b980-b0a792d6a695.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=476ed07d-a35d-43f6-9ac3-b9aff31b20f1 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:48.324Z InvocationID=ec48b520-a6e1-4140-a76f-043bc512a1ff ErrorCode=500 Ended create Csar for component VDU VirtualObjectStorage 2022-04-26T06:24:49.290Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "7aaf628f-8a3e-47d8-9aa0-f4a0050c7ee6" INVARIANT_UUID = "edc6bf35-e9c8-497d-b8c6-55dde16f9f26" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "09c69ff1-e322-468c-b980-b0a792d6a695.assettoscatemplate" CURR_ARTIFACT_UUID = "09c69ff1-e322-468c-b980-b0a792d6a695.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualobjectstorage-template.yml,0,09c69ff1-e322-468c-b980-b0a792d6a695.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=476ed07d-a35d-43f6-9ac3-b9aff31b20f1 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:48.324Z InvocationID=ec48b520-a6e1-4140-a76f-043bc512a1ff ErrorCode=500 lock resource with id 09c69ff1-e322-468c-b980-b0a792d6a695 2022-04-26T06:24:49.833Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=5c1a14c4-1ee9-479b-9dc1-c48bc22ff194 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:24:49.833Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=5c1a14c4-1ee9-479b-9dc1-c48bc22ff194 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:24:49.945Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "7aaf628f-8a3e-47d8-9aa0-f4a0050c7ee6" INVARIANT_UUID = "edc6bf35-e9c8-497d-b8c6-55dde16f9f26" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "09c69ff1-e322-468c-b980-b0a792d6a695.assettoscacsar" CURR_ARTIFACT_UUID = "09c69ff1-e322-468c-b980-b0a792d6a695.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduVirtualobjectstorage-csar.csar,0,09c69ff1-e322-468c-b980-b0a792d6a695.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1620 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0e660ab1-604d-4684-94d9-563a2c096fac TargetServiceName=null EntryTimestamp=2022-04-26T06:24:48.324Z InvocationID=ec48b520-a6e1-4140-a76f-043bc512a1ff ErrorCode=500 LogTimestamp=2022-04-26T06:24:49.944397Z ResponseDesc=Created Exiting. 2022-04-26T06:24:49.946Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:24:49.959Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1c7fab05-2469-4f93-a15c-4cb771a19f9c InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:49.958667Z InvocationID=b9118a17-4c7c-4d21-9cc9-9fc9e16e6fe7 LogTimestamp=2022-04-26T06:24:49.958736Z Entering 2022-04-26T06:24:49.959Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1c7fab05-2469-4f93-a15c-4cb771a19f9c InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:49.958667Z InvocationID=b9118a17-4c7c-4d21-9cc9-9fc9e16e6fe7 LogTimestamp=2022-04-26T06:24:49.958736Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:24:50.349Z [qtp2085002312-14] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Import Resource vduVirtualStorage.yml ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:24:49.959Z InvocationID=559fe9c7-e636-4758-bb45-17e2820017aa ErrorCode=500 Unexpected error during operation 2022-04-26T06:24:50.355Z [qtp2085002312-14] ERROR o.o.s.b.s.ResourceUploadServlet AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "" INVARIANT_UUID = "" PREV_VERSION = "1.0" CURR_VERSION = "" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "" STATUS = "409" DESC = "SVC4152: Error: Component 'VDU VirtualStorage' with Vendor Release '1.0.0.wd03' already exists." TOSCA_NODE_TYPE = "" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Import Resource vduVirtualStorage.yml ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:24:49.959Z InvocationID=559fe9c7-e636-4758-bb45-17e2820017aa ErrorCode=500 #uploadMultipart('multipart', /tmp/rep8163664050799177708tmp, 'form-data; filename="vduVirtualStorage.zip"; name="resourceZip"', '{"payloadName": "vduVirtualStorage.yml", "contactId": "jh0003", "name": "VDU VirtualStorage", "description": "VDU VirtualStorage", "resourceIconPath": "objectStorage", "resourceType": "VFC", "categories": [{"name": "Generic", "subcategories": [{"name": "Infrastructure"}]}], "tags": ["VDU VirtualStorage"]}', 'Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/catalog/upload/multipart?createNewVersion=true)@4e1a4d00', 'jh0003', true): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.components.impl.ResourceImportManager#checkResourceExistsOnCreate[263] in 393ms 2022-04-26T06:24:50.356Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "" INVARIANT_UUID = "" PREV_VERSION = "1.0" CURR_VERSION = "" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "" STATUS = "409" DESC = "SVC4152: Error: Component 'VDU VirtualStorage' with Vendor Release '1.0.0.wd03' already exists." TOSCA_NODE_TYPE = "" ResponseCode=409 RequestID=null ElapsedTime=397 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=ERROR TargetEntity=Import Resource vduVirtualStorage.yml ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null ErrorDesc=Conflict EntryTimestamp=2022-04-26T06:24:49.959Z InvocationID=559fe9c7-e636-4758-bb45-17e2820017aa ErrorCode=409 LogTimestamp=2022-04-26T06:24:50.356392Z ResponseDesc=Conflict Exiting. 2022-04-26T06:24:50.357Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="409" 2022-04-26T06:24:50.364Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=537f58ce-58ac-4cd6-80b7-abb427dabe6a InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:50.364110Z InvocationID=01cd6ec1-23f7-445d-ba16-6f2cc1e4dbdd LogTimestamp=2022-04-26T06:24:50.364168Z Entering 2022-04-26T06:24:50.364Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=537f58ce-58ac-4cd6-80b7-abb427dabe6a InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:50.364110Z InvocationID=01cd6ec1-23f7-445d-ba16-6f2cc1e4dbdd LogTimestamp=2022-04-26T06:24:50.364168Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:24:50.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:24:50.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:24:50.753Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:50.365Z InvocationID=3d52392d-d13a-445e-99b5-614f213fecc9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:50.753Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:50.365Z InvocationID=3d52392d-d13a-445e-99b5-614f213fecc9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:50.753Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:50.365Z InvocationID=3d52392d-d13a-445e-99b5-614f213fecc9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:50.753Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:50.365Z InvocationID=3d52392d-d13a-445e-99b5-614f213fecc9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:50.758Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:50.365Z InvocationID=3d52392d-d13a-445e-99b5-614f213fecc9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Vnfvirtuallink 2022-04-26T06:24:51.050Z [qtp2085002312-15] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VnfVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "c6934259-3eb6-4356-a651-64149dcf95b2" INVARIANT_UUID = "6940f7bc-e505-4f30-b693-de8abad5c3af" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VnfVirtualLink" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c6934259-3eb6-4356-a651-64149dcf95b2 EntryTimestamp=2022-04-26T06:24:50.365Z InvocationID=3d52392d-d13a-445e-99b5-614f213fecc9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:24:51.051Z [qtp2085002312-15] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VnfVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "c6934259-3eb6-4356-a651-64149dcf95b2" INVARIANT_UUID = "6940f7bc-e505-4f30-b693-de8abad5c3af" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VnfVirtualLink" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c6934259-3eb6-4356-a651-64149dcf95b2 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:50.365Z InvocationID=3d52392d-d13a-445e-99b5-614f213fecc9 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:24:51.348Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VnfVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "c6934259-3eb6-4356-a651-64149dcf95b2" INVARIANT_UUID = "6940f7bc-e505-4f30-b693-de8abad5c3af" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VnfVirtualLink" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c6934259-3eb6-4356-a651-64149dcf95b2 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:50.365Z InvocationID=3d52392d-d13a-445e-99b5-614f213fecc9 ErrorCode=500 lock resource with id 5cacec7d-4f1a-4394-8512-622f35b3a729 2022-04-26T06:24:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:24:52.686Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:24:52.686Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:24:52.740Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=103f45f0-b9bf-4c9b-be47-2d4be9f9297b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:24:52.740Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=103f45f0-b9bf-4c9b-be47-2d4be9f9297b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3d5a6bd8 2022-04-26T06:24:52.740Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=103f45f0-b9bf-4c9b-be47-2d4be9f9297b PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:24:52.740Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=103f45f0-b9bf-4c9b-be47-2d4be9f9297b PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:24:52.953Z [qtp2085002312-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VnfVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "c6934259-3eb6-4356-a651-64149dcf95b2" INVARIANT_UUID = "6940f7bc-e505-4f30-b693-de8abad5c3af" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5cacec7d-4f1a-4394-8512-622f35b3a729.assettoscatemplate" CURR_ARTIFACT_UUID = "5cacec7d-4f1a-4394-8512-622f35b3a729.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnfvirtuallink-template.yml,0,5cacec7d-4f1a-4394-8512-622f35b3a729.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=fca20234-c9e0-49bf-b1f2-e40a6ed23623 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:50.365Z InvocationID=3d52392d-d13a-445e-99b5-614f213fecc9 ErrorCode=500 Starting to create Csar for component VnfVirtualLink 2022-04-26T06:24:53.110Z [qtp2085002312-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VnfVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "c6934259-3eb6-4356-a651-64149dcf95b2" INVARIANT_UUID = "6940f7bc-e505-4f30-b693-de8abad5c3af" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5cacec7d-4f1a-4394-8512-622f35b3a729.assettoscatemplate" CURR_ARTIFACT_UUID = "5cacec7d-4f1a-4394-8512-622f35b3a729.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnfvirtuallink-template.yml,0,5cacec7d-4f1a-4394-8512-622f35b3a729.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=fca20234-c9e0-49bf-b1f2-e40a6ed23623 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:50.365Z InvocationID=3d52392d-d13a-445e-99b5-614f213fecc9 ErrorCode=500 Ended create Csar for component VnfVirtualLink 2022-04-26T06:24:53.111Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VnfVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "c6934259-3eb6-4356-a651-64149dcf95b2" INVARIANT_UUID = "6940f7bc-e505-4f30-b693-de8abad5c3af" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5cacec7d-4f1a-4394-8512-622f35b3a729.assettoscatemplate" CURR_ARTIFACT_UUID = "5cacec7d-4f1a-4394-8512-622f35b3a729.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnfvirtuallink-template.yml,0,5cacec7d-4f1a-4394-8512-622f35b3a729.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=fca20234-c9e0-49bf-b1f2-e40a6ed23623 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:50.365Z InvocationID=3d52392d-d13a-445e-99b5-614f213fecc9 ErrorCode=500 lock resource with id 5cacec7d-4f1a-4394-8512-622f35b3a729 2022-04-26T06:24:53.764Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VnfVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "c6934259-3eb6-4356-a651-64149dcf95b2" INVARIANT_UUID = "6940f7bc-e505-4f30-b693-de8abad5c3af" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5cacec7d-4f1a-4394-8512-622f35b3a729.assettoscacsar" CURR_ARTIFACT_UUID = "5cacec7d-4f1a-4394-8512-622f35b3a729.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Vnfvirtuallink-csar.csar,0,5cacec7d-4f1a-4394-8512-622f35b3a729.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=3399 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6e13d352-31d8-452a-965f-a1da494f661d TargetServiceName=null EntryTimestamp=2022-04-26T06:24:50.365Z InvocationID=3d52392d-d13a-445e-99b5-614f213fecc9 ErrorCode=500 LogTimestamp=2022-04-26T06:24:53.764548Z ResponseDesc=Created Exiting. 2022-04-26T06:24:53.765Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:24:53.768Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=103f45f0-b9bf-4c9b-be47-2d4be9f9297b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:24:53.769Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=103f45f0-b9bf-4c9b-be47-2d4be9f9297b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:24:53.844Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ea35c72e-e291-4a7e-821d-bafc051affc6 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:53.844167Z InvocationID=c626f246-f179-4d1e-9179-e6bd0c99ed08 LogTimestamp=2022-04-26T06:24:53.844236Z Entering 2022-04-26T06:24:53.844Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ea35c72e-e291-4a7e-821d-bafc051affc6 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:53.844167Z InvocationID=c626f246-f179-4d1e-9179-e6bd0c99ed08 LogTimestamp=2022-04-26T06:24:53.844236Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:24:54.455Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:53.845Z InvocationID=ee17c0a9-43f0-4e3d-afd2-d4120a59f547 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:54.456Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:53.845Z InvocationID=ee17c0a9-43f0-4e3d-afd2-d4120a59f547 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:54.456Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:53.845Z InvocationID=ee17c0a9-43f0-4e3d-afd2-d4120a59f547 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:54.456Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:53.845Z InvocationID=ee17c0a9-43f0-4e3d-afd2-d4120a59f547 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:54.539Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:53.845Z InvocationID=ee17c0a9-43f0-4e3d-afd2-d4120a59f547 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name VnfExternalCp 2022-04-26T06:24:54.854Z [qtp2085002312-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "b394fa2f-70a1-4f72-83aa-a8dd4d0613e6" INVARIANT_UUID = "20e54cf4-2d55-4e9b-9d10-440bb37b149c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VnfExtCp" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b394fa2f-70a1-4f72-83aa-a8dd4d0613e6 EntryTimestamp=2022-04-26T06:24:53.845Z InvocationID=ee17c0a9-43f0-4e3d-afd2-d4120a59f547 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:24:54.855Z [qtp2085002312-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "b394fa2f-70a1-4f72-83aa-a8dd4d0613e6" INVARIANT_UUID = "20e54cf4-2d55-4e9b-9d10-440bb37b149c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VnfExtCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b394fa2f-70a1-4f72-83aa-a8dd4d0613e6 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:53.845Z InvocationID=ee17c0a9-43f0-4e3d-afd2-d4120a59f547 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:24:55.152Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "b394fa2f-70a1-4f72-83aa-a8dd4d0613e6" INVARIANT_UUID = "20e54cf4-2d55-4e9b-9d10-440bb37b149c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VnfExtCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b394fa2f-70a1-4f72-83aa-a8dd4d0613e6 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:53.845Z InvocationID=ee17c0a9-43f0-4e3d-afd2-d4120a59f547 ErrorCode=500 lock resource with id 5e77c04e-396c-48b0-93af-09b775df8cb0 2022-04-26T06:24:55.845Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e935cfe3-4baa-4835-86b4-f4476b4d2a4d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:24:55.845Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e935cfe3-4baa-4835-86b4-f4476b4d2a4d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2be783c7 2022-04-26T06:24:55.845Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e935cfe3-4baa-4835-86b4-f4476b4d2a4d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:24:55.845Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e935cfe3-4baa-4835-86b4-f4476b4d2a4d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:24:56.350Z [qtp2085002312-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "b394fa2f-70a1-4f72-83aa-a8dd4d0613e6" INVARIANT_UUID = "20e54cf4-2d55-4e9b-9d10-440bb37b149c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5e77c04e-396c-48b0-93af-09b775df8cb0.assettoscatemplate" CURR_ARTIFACT_UUID = "5e77c04e-396c-48b0-93af-09b775df8cb0.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VnfExternalCp-template.yml,0,5e77c04e-396c-48b0-93af-09b775df8cb0.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3892d571-cc1e-4e3d-b3b8-78d2f92f348a TargetServiceName=null EntryTimestamp=2022-04-26T06:24:53.845Z InvocationID=ee17c0a9-43f0-4e3d-afd2-d4120a59f547 ErrorCode=500 Starting to create Csar for component VNF External CP 2022-04-26T06:24:56.555Z [qtp2085002312-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "b394fa2f-70a1-4f72-83aa-a8dd4d0613e6" INVARIANT_UUID = "20e54cf4-2d55-4e9b-9d10-440bb37b149c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5e77c04e-396c-48b0-93af-09b775df8cb0.assettoscatemplate" CURR_ARTIFACT_UUID = "5e77c04e-396c-48b0-93af-09b775df8cb0.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VnfExternalCp-template.yml,0,5e77c04e-396c-48b0-93af-09b775df8cb0.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3892d571-cc1e-4e3d-b3b8-78d2f92f348a TargetServiceName=null EntryTimestamp=2022-04-26T06:24:53.845Z InvocationID=ee17c0a9-43f0-4e3d-afd2-d4120a59f547 ErrorCode=500 Ended create Csar for component VNF External CP 2022-04-26T06:24:56.556Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "b394fa2f-70a1-4f72-83aa-a8dd4d0613e6" INVARIANT_UUID = "20e54cf4-2d55-4e9b-9d10-440bb37b149c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5e77c04e-396c-48b0-93af-09b775df8cb0.assettoscatemplate" CURR_ARTIFACT_UUID = "5e77c04e-396c-48b0-93af-09b775df8cb0.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VnfExternalCp-template.yml,0,5e77c04e-396c-48b0-93af-09b775df8cb0.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3892d571-cc1e-4e3d-b3b8-78d2f92f348a TargetServiceName=null EntryTimestamp=2022-04-26T06:24:53.845Z InvocationID=ee17c0a9-43f0-4e3d-afd2-d4120a59f547 ErrorCode=500 lock resource with id 5e77c04e-396c-48b0-93af-09b775df8cb0 2022-04-26T06:24:56.874Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e935cfe3-4baa-4835-86b4-f4476b4d2a4d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:24:56.875Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e935cfe3-4baa-4835-86b4-f4476b4d2a4d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:24:57.453Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "b394fa2f-70a1-4f72-83aa-a8dd4d0613e6" INVARIANT_UUID = "20e54cf4-2d55-4e9b-9d10-440bb37b149c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5e77c04e-396c-48b0-93af-09b775df8cb0.assettoscacsar" CURR_ARTIFACT_UUID = "5e77c04e-396c-48b0-93af-09b775df8cb0.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VnfExternalCp-csar.csar,0,5e77c04e-396c-48b0-93af-09b775df8cb0.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=3608 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=de9666f2-30ec-43a2-a6b1-d874962d9143 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:53.845Z InvocationID=ee17c0a9-43f0-4e3d-afd2-d4120a59f547 ErrorCode=500 LogTimestamp=2022-04-26T06:24:57.453623Z ResponseDesc=Created Exiting. 2022-04-26T06:24:57.454Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:24:57.467Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0d39879e-c6eb-4a86-8202-6374477a02bd InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:57.467090Z InvocationID=b45fd06a-e881-4c10-a0ad-b31a2dd368d3 LogTimestamp=2022-04-26T06:24:57.467146Z Entering 2022-04-26T06:24:57.467Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0d39879e-c6eb-4a86-8202-6374477a02bd InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:24:57.467090Z InvocationID=b45fd06a-e881-4c10-a0ad-b31a2dd368d3 LogTimestamp=2022-04-26T06:24:57.467146Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:24:58.056Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:57.467Z InvocationID=d767113c-f655-46d7-81fb-0331e1073fc9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:58.056Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:57.467Z InvocationID=d767113c-f655-46d7-81fb-0331e1073fc9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:58.057Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:57.467Z InvocationID=d767113c-f655-46d7-81fb-0331e1073fc9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:58.057Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:57.467Z InvocationID=d767113c-f655-46d7-81fb-0331e1073fc9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:24:58.065Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:24:57.467Z InvocationID=d767113c-f655-46d7-81fb-0331e1073fc9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name VduCp 2022-04-26T06:24:58.251Z [qtp2085002312-20] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3046d067-ef25-4477-a194-ba2b5d40f677" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VduCp" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3046d067-ef25-4477-a194-ba2b5d40f677 EntryTimestamp=2022-04-26T06:24:57.467Z InvocationID=d767113c-f655-46d7-81fb-0331e1073fc9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:24:58.251Z [qtp2085002312-20] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3046d067-ef25-4477-a194-ba2b5d40f677" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VduCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3046d067-ef25-4477-a194-ba2b5d40f677 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:57.467Z InvocationID=d767113c-f655-46d7-81fb-0331e1073fc9 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:24:58.563Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3046d067-ef25-4477-a194-ba2b5d40f677" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VduCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3046d067-ef25-4477-a194-ba2b5d40f677 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:57.467Z InvocationID=d767113c-f655-46d7-81fb-0331e1073fc9 ErrorCode=500 lock resource with id 6c919be2-e998-47c6-af28-39cdf81a48d7 2022-04-26T06:24:58.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ab30c7f-e117-4dcd-9da0-1b7623f3f3b5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:24:58.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ab30c7f-e117-4dcd-9da0-1b7623f3f3b5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@317fa963 2022-04-26T06:24:58.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ab30c7f-e117-4dcd-9da0-1b7623f3f3b5 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:24:58.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ab30c7f-e117-4dcd-9da0-1b7623f3f3b5 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:24:59.693Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=0ab30c7f-e117-4dcd-9da0-1b7623f3f3b5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:24:59.693Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=0ab30c7f-e117-4dcd-9da0-1b7623f3f3b5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:25:00.843Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3046d067-ef25-4477-a194-ba2b5d40f677" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "6c919be2-e998-47c6-af28-39cdf81a48d7.assettoscatemplate" CURR_ARTIFACT_UUID = "6c919be2-e998-47c6-af28-39cdf81a48d7.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCp-template.yml,0,6c919be2-e998-47c6-af28-39cdf81a48d7.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=14400f60-98f4-44b7-9b44-ed0433e547f7 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:57.467Z InvocationID=d767113c-f655-46d7-81fb-0331e1073fc9 ErrorCode=500 Starting to create Csar for component VDU Cp 2022-04-26T06:25:00.998Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3046d067-ef25-4477-a194-ba2b5d40f677" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "6c919be2-e998-47c6-af28-39cdf81a48d7.assettoscatemplate" CURR_ARTIFACT_UUID = "6c919be2-e998-47c6-af28-39cdf81a48d7.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCp-template.yml,0,6c919be2-e998-47c6-af28-39cdf81a48d7.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=14400f60-98f4-44b7-9b44-ed0433e547f7 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:57.467Z InvocationID=d767113c-f655-46d7-81fb-0331e1073fc9 ErrorCode=500 Ended create Csar for component VDU Cp 2022-04-26T06:25:00.998Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3046d067-ef25-4477-a194-ba2b5d40f677" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "6c919be2-e998-47c6-af28-39cdf81a48d7.assettoscatemplate" CURR_ARTIFACT_UUID = "6c919be2-e998-47c6-af28-39cdf81a48d7.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCp-template.yml,0,6c919be2-e998-47c6-af28-39cdf81a48d7.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=14400f60-98f4-44b7-9b44-ed0433e547f7 TargetServiceName=null EntryTimestamp=2022-04-26T06:24:57.467Z InvocationID=d767113c-f655-46d7-81fb-0331e1073fc9 ErrorCode=500 lock resource with id 6c919be2-e998-47c6-af28-39cdf81a48d7 2022-04-26T06:25:01.732Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48a01660-b6cf-4e29-b374-fbc322cac573 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:25:01.732Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48a01660-b6cf-4e29-b374-fbc322cac573 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@19bd67c1 2022-04-26T06:25:01.733Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48a01660-b6cf-4e29-b374-fbc322cac573 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:25:01.733Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48a01660-b6cf-4e29-b374-fbc322cac573 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:25:02.350Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3046d067-ef25-4477-a194-ba2b5d40f677" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "6c919be2-e998-47c6-af28-39cdf81a48d7.assettoscacsar" CURR_ARTIFACT_UUID = "6c919be2-e998-47c6-af28-39cdf81a48d7.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduCp-csar.csar,0,6c919be2-e998-47c6-af28-39cdf81a48d7.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=4883 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bd6465db-1f4c-40f2-9dd2-3681b223c0dc TargetServiceName=null EntryTimestamp=2022-04-26T06:24:57.467Z InvocationID=d767113c-f655-46d7-81fb-0331e1073fc9 ErrorCode=500 LogTimestamp=2022-04-26T06:25:02.350380Z ResponseDesc=Created Exiting. 2022-04-26T06:25:02.351Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:25:02.444Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=dc7c89e7-6872-4fa5-9b35-ed37fb60839c InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:02.444201Z InvocationID=3338fba5-b1c1-4b8f-9c3a-a20c7de99bda LogTimestamp=2022-04-26T06:25:02.444273Z Entering 2022-04-26T06:25:02.445Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=dc7c89e7-6872-4fa5-9b35-ed37fb60839c InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:02.444201Z InvocationID=3338fba5-b1c1-4b8f-9c3a-a20c7de99bda LogTimestamp=2022-04-26T06:25:02.444273Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:25:02.761Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=48a01660-b6cf-4e29-b374-fbc322cac573 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:25:02.761Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=48a01660-b6cf-4e29-b374-fbc322cac573 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:25:03.855Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:02.446Z InvocationID=af052b25-6e9a-4e2a-a26d-08e5011cb118 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:03.856Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:02.446Z InvocationID=af052b25-6e9a-4e2a-a26d-08e5011cb118 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:03.856Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:02.446Z InvocationID=af052b25-6e9a-4e2a-a26d-08e5011cb118 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:03.857Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:02.446Z InvocationID=af052b25-6e9a-4e2a-a26d-08e5011cb118 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:03.933Z [qtp2085002312-18] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Create Resource - validateLifecycleTypesCreate ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:25:02.446Z InvocationID=af052b25-6e9a-4e2a-a26d-08e5011cb118 ErrorCode=400 Interface tosca.interfaces.nfv.vnflcm required is missing 2022-04-26T06:25:03.934Z [qtp2085002312-18] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Create Resource - validateLifecycleTypesCreate ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:25:02.446Z InvocationID=af052b25-6e9a-4e2a-a26d-08e5011cb118 ErrorCode=500 Operation towards database failed. 2022-04-26T06:25:03.934Z [qtp2085002312-18] WARN o.o.s.e.AbstractSdncException RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:25:02.446Z InvocationID=af052b25-6e9a-4e2a-a26d-08e5011cb118 ErrorCode=900 2022-04-26T06:25:03.938Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "" INVARIANT_UUID = "" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "400" DESC = "SVC4134: Error: Invalid Content. Missing interface life-cycle type." TOSCA_NODE_TYPE = "tosca.nodes.nfv.VNF" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:25:02.446Z InvocationID=af052b25-6e9a-4e2a-a26d-08e5011cb118 ErrorCode=900 lock resource with name Vnf 2022-04-26T06:25:04.341Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=229a4792-06c8-4c8e-96cd-010cecf1d368 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:25:04.342Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=229a4792-06c8-4c8e-96cd-010cecf1d368 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@22c81b0d 2022-04-26T06:25:04.342Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=229a4792-06c8-4c8e-96cd-010cecf1d368 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:25:04.342Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=229a4792-06c8-4c8e-96cd-010cecf1d368 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:25:04.430Z [qtp2085002312-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1203db9d-6627-4d55-b8c8-376bf645b3f9" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VNF" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1203db9d-6627-4d55-b8c8-376bf645b3f9 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:02.446Z InvocationID=af052b25-6e9a-4e2a-a26d-08e5011cb118 ErrorCode=900 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:04.431Z [qtp2085002312-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1203db9d-6627-4d55-b8c8-376bf645b3f9" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VNF" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1203db9d-6627-4d55-b8c8-376bf645b3f9 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:02.446Z InvocationID=af052b25-6e9a-4e2a-a26d-08e5011cb118 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:04.757Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1203db9d-6627-4d55-b8c8-376bf645b3f9" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VNF" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1203db9d-6627-4d55-b8c8-376bf645b3f9 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:02.446Z InvocationID=af052b25-6e9a-4e2a-a26d-08e5011cb118 ErrorCode=500 lock resource with id 97287295-ddf7-4f36-aa8f-299f9c2b7846 2022-04-26T06:25:05.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:25:05.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:25:05.389Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=229a4792-06c8-4c8e-96cd-010cecf1d368 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:25:05.389Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=229a4792-06c8-4c8e-96cd-010cecf1d368 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:25:07.135Z [qtp2085002312-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1203db9d-6627-4d55-b8c8-376bf645b3f9" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "97287295-ddf7-4f36-aa8f-299f9c2b7846.assettoscatemplate" CURR_ARTIFACT_UUID = "97287295-ddf7-4f36-aa8f-299f9c2b7846.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnf-template.yml,0,97287295-ddf7-4f36-aa8f-299f9c2b7846.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4316bcf0-bcaa-4ee9-a322-cc7dcc7cacd8 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:02.446Z InvocationID=af052b25-6e9a-4e2a-a26d-08e5011cb118 ErrorCode=500 Starting to create Csar for component VNF 2022-04-26T06:25:07.303Z [qtp2085002312-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1203db9d-6627-4d55-b8c8-376bf645b3f9" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "97287295-ddf7-4f36-aa8f-299f9c2b7846.assettoscatemplate" CURR_ARTIFACT_UUID = "97287295-ddf7-4f36-aa8f-299f9c2b7846.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnf-template.yml,0,97287295-ddf7-4f36-aa8f-299f9c2b7846.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4316bcf0-bcaa-4ee9-a322-cc7dcc7cacd8 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:02.446Z InvocationID=af052b25-6e9a-4e2a-a26d-08e5011cb118 ErrorCode=500 Ended create Csar for component VNF 2022-04-26T06:25:07.303Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1203db9d-6627-4d55-b8c8-376bf645b3f9" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "97287295-ddf7-4f36-aa8f-299f9c2b7846.assettoscatemplate" CURR_ARTIFACT_UUID = "97287295-ddf7-4f36-aa8f-299f9c2b7846.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnf-template.yml,0,97287295-ddf7-4f36-aa8f-299f9c2b7846.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4316bcf0-bcaa-4ee9-a322-cc7dcc7cacd8 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:02.446Z InvocationID=af052b25-6e9a-4e2a-a26d-08e5011cb118 ErrorCode=500 lock resource with id 97287295-ddf7-4f36-aa8f-299f9c2b7846 2022-04-26T06:25:07.535Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2fddbc7-70cc-44bf-bd53-d30f92af8243 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:25:07.535Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2fddbc7-70cc-44bf-bd53-d30f92af8243 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@47de8924 2022-04-26T06:25:07.536Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2fddbc7-70cc-44bf-bd53-d30f92af8243 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:25:07.536Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2fddbc7-70cc-44bf-bd53-d30f92af8243 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:25:07.650Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:25:08.255Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1203db9d-6627-4d55-b8c8-376bf645b3f9" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "97287295-ddf7-4f36-aa8f-299f9c2b7846.assettoscacsar" CURR_ARTIFACT_UUID = "97287295-ddf7-4f36-aa8f-299f9c2b7846.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Vnf-csar.csar,0,97287295-ddf7-4f36-aa8f-299f9c2b7846.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5809 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=cca821ae-ed16-4e00-8260-2b8078f600ef TargetServiceName=null EntryTimestamp=2022-04-26T06:25:02.446Z InvocationID=af052b25-6e9a-4e2a-a26d-08e5011cb118 ErrorCode=500 LogTimestamp=2022-04-26T06:25:08.255042Z ResponseDesc=Created Exiting. 2022-04-26T06:25:08.255Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:25:08.334Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c247f61e-e532-46f6-998a-1e6eef8120c7 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:08.333730Z InvocationID=e84b392b-55d9-47a5-aa11-0f0737823e4f LogTimestamp=2022-04-26T06:25:08.333810Z Entering 2022-04-26T06:25:08.334Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c247f61e-e532-46f6-998a-1e6eef8120c7 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:08.333730Z InvocationID=e84b392b-55d9-47a5-aa11-0f0737823e4f LogTimestamp=2022-04-26T06:25:08.333810Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:25:08.558Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e2fddbc7-70cc-44bf-bd53-d30f92af8243 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:25:08.559Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e2fddbc7-70cc-44bf-bd53-d30f92af8243 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:25:08.931Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:08.334Z InvocationID=dc90c6e4-8920-4b19-ad40-82e9a9aca57e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:08.932Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:08.334Z InvocationID=dc90c6e4-8920-4b19-ad40-82e9a9aca57e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:08.932Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:08.334Z InvocationID=dc90c6e4-8920-4b19-ad40-82e9a9aca57e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:08.932Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:08.334Z InvocationID=dc90c6e4-8920-4b19-ad40-82e9a9aca57e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:08.935Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:08.334Z InvocationID=dc90c6e4-8920-4b19-ad40-82e9a9aca57e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name AccessConnectivity 2022-04-26T06:25:09.134Z [qtp2085002312-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Access Connectivity" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "86dc0657-52e8-42d9-8a56-ab10a7618a0e" INVARIANT_UUID = "ea03cee8-c77c-4fe7-9210-6a7d59eed20a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.accessConnectivity" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=86dc0657-52e8-42d9-8a56-ab10a7618a0e EntryTimestamp=2022-04-26T06:25:08.334Z InvocationID=dc90c6e4-8920-4b19-ad40-82e9a9aca57e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:09.135Z [qtp2085002312-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Access Connectivity" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "86dc0657-52e8-42d9-8a56-ab10a7618a0e" INVARIANT_UUID = "ea03cee8-c77c-4fe7-9210-6a7d59eed20a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.accessConnectivity" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=86dc0657-52e8-42d9-8a56-ab10a7618a0e TargetServiceName=null EntryTimestamp=2022-04-26T06:25:08.334Z InvocationID=dc90c6e4-8920-4b19-ad40-82e9a9aca57e ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:09.355Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Access Connectivity" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "86dc0657-52e8-42d9-8a56-ab10a7618a0e" INVARIANT_UUID = "ea03cee8-c77c-4fe7-9210-6a7d59eed20a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.accessConnectivity" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=86dc0657-52e8-42d9-8a56-ab10a7618a0e TargetServiceName=null EntryTimestamp=2022-04-26T06:25:08.334Z InvocationID=dc90c6e4-8920-4b19-ad40-82e9a9aca57e ErrorCode=500 lock resource with id f605e728-f0d8-4d80-bd4e-a53df9197a04 2022-04-26T06:25:09.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5afee80b-427f-42c0-9b43-9f1326981d97 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:25:09.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5afee80b-427f-42c0-9b43-9f1326981d97 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4fee84c0 2022-04-26T06:25:09.880Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5afee80b-427f-42c0-9b43-9f1326981d97 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:25:09.880Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5afee80b-427f-42c0-9b43-9f1326981d97 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:25:10.182Z [qtp2085002312-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Access Connectivity" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "86dc0657-52e8-42d9-8a56-ab10a7618a0e" INVARIANT_UUID = "ea03cee8-c77c-4fe7-9210-6a7d59eed20a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "f605e728-f0d8-4d80-bd4e-a53df9197a04.assettoscatemplate" CURR_ARTIFACT_UUID = "f605e728-f0d8-4d80-bd4e-a53df9197a04.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-AccessConnectivity-template.yml,0,f605e728-f0d8-4d80-bd4e-a53df9197a04.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ebdb77b2-e985-43ce-8c27-51c135f964c1 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:08.334Z InvocationID=dc90c6e4-8920-4b19-ad40-82e9a9aca57e ErrorCode=500 Starting to create Csar for component Access Connectivity 2022-04-26T06:25:10.381Z [qtp2085002312-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Access Connectivity" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "86dc0657-52e8-42d9-8a56-ab10a7618a0e" INVARIANT_UUID = "ea03cee8-c77c-4fe7-9210-6a7d59eed20a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "f605e728-f0d8-4d80-bd4e-a53df9197a04.assettoscatemplate" CURR_ARTIFACT_UUID = "f605e728-f0d8-4d80-bd4e-a53df9197a04.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-AccessConnectivity-template.yml,0,f605e728-f0d8-4d80-bd4e-a53df9197a04.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ebdb77b2-e985-43ce-8c27-51c135f964c1 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:08.334Z InvocationID=dc90c6e4-8920-4b19-ad40-82e9a9aca57e ErrorCode=500 Ended create Csar for component Access Connectivity 2022-04-26T06:25:10.382Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Access Connectivity" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "86dc0657-52e8-42d9-8a56-ab10a7618a0e" INVARIANT_UUID = "ea03cee8-c77c-4fe7-9210-6a7d59eed20a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "f605e728-f0d8-4d80-bd4e-a53df9197a04.assettoscatemplate" CURR_ARTIFACT_UUID = "f605e728-f0d8-4d80-bd4e-a53df9197a04.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-AccessConnectivity-template.yml,0,f605e728-f0d8-4d80-bd4e-a53df9197a04.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ebdb77b2-e985-43ce-8c27-51c135f964c1 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:08.334Z InvocationID=dc90c6e4-8920-4b19-ad40-82e9a9aca57e ErrorCode=500 lock resource with id f605e728-f0d8-4d80-bd4e-a53df9197a04 2022-04-26T06:25:10.893Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=5afee80b-427f-42c0-9b43-9f1326981d97 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:25:10.894Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=5afee80b-427f-42c0-9b43-9f1326981d97 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:25:10.942Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Access Connectivity" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "86dc0657-52e8-42d9-8a56-ab10a7618a0e" INVARIANT_UUID = "ea03cee8-c77c-4fe7-9210-6a7d59eed20a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "f605e728-f0d8-4d80-bd4e-a53df9197a04.assettoscacsar" CURR_ARTIFACT_UUID = "f605e728-f0d8-4d80-bd4e-a53df9197a04.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-AccessConnectivity-csar.csar,0,f605e728-f0d8-4d80-bd4e-a53df9197a04.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=2608 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d8eea2f3-938b-4788-bd9b-14c0c9069b73 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:08.334Z InvocationID=dc90c6e4-8920-4b19-ad40-82e9a9aca57e ErrorCode=500 LogTimestamp=2022-04-26T06:25:10.942513Z ResponseDesc=Created Exiting. 2022-04-26T06:25:10.943Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:25:10.962Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=9e34b12b-5a5e-4002-88f4-5f54adb90462 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:10.962372Z InvocationID=fdcce697-96ba-4944-9fcd-a34f0304a79b LogTimestamp=2022-04-26T06:25:10.962517Z Entering 2022-04-26T06:25:10.963Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=9e34b12b-5a5e-4002-88f4-5f54adb90462 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:10.962372Z InvocationID=fdcce697-96ba-4944-9fcd-a34f0304a79b LogTimestamp=2022-04-26T06:25:10.962517Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:25:11.061Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:10.963Z InvocationID=a69508e4-c76c-4e74-95a4-b248874d29d5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:11.062Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:10.963Z InvocationID=a69508e4-c76c-4e74-95a4-b248874d29d5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:11.062Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:10.963Z InvocationID=a69508e4-c76c-4e74-95a4-b248874d29d5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:11.062Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:10.963Z InvocationID=a69508e4-c76c-4e74-95a4-b248874d29d5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:11.066Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:10.963Z InvocationID=a69508e4-c76c-4e74-95a4-b248874d29d5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Ontpnf 2022-04-26T06:25:11.104Z [qtp2085002312-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "OntPnf" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a80ba61f-597e-414e-82f9-6d34548fc61d" INVARIANT_UUID = "634ff3dc-bab6-4d05-953d-6c8f2a107fa8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.OntPnf" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a80ba61f-597e-414e-82f9-6d34548fc61d EntryTimestamp=2022-04-26T06:25:10.963Z InvocationID=a69508e4-c76c-4e74-95a4-b248874d29d5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:11.105Z [qtp2085002312-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "OntPnf" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a80ba61f-597e-414e-82f9-6d34548fc61d" INVARIANT_UUID = "634ff3dc-bab6-4d05-953d-6c8f2a107fa8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.OntPnf" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a80ba61f-597e-414e-82f9-6d34548fc61d TargetServiceName=null EntryTimestamp=2022-04-26T06:25:10.963Z InvocationID=a69508e4-c76c-4e74-95a4-b248874d29d5 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:11.136Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "OntPnf" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a80ba61f-597e-414e-82f9-6d34548fc61d" INVARIANT_UUID = "634ff3dc-bab6-4d05-953d-6c8f2a107fa8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.OntPnf" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a80ba61f-597e-414e-82f9-6d34548fc61d TargetServiceName=null EntryTimestamp=2022-04-26T06:25:10.963Z InvocationID=a69508e4-c76c-4e74-95a4-b248874d29d5 ErrorCode=500 lock resource with id 93edbabf-e707-43bc-b852-41cb0129f511 2022-04-26T06:25:11.753Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "OntPnf" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a80ba61f-597e-414e-82f9-6d34548fc61d" INVARIANT_UUID = "634ff3dc-bab6-4d05-953d-6c8f2a107fa8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "93edbabf-e707-43bc-b852-41cb0129f511.assettoscatemplate" CURR_ARTIFACT_UUID = "93edbabf-e707-43bc-b852-41cb0129f511.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ontpnf-template.yml,0,93edbabf-e707-43bc-b852-41cb0129f511.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7187a394-3105-4001-bace-262cb2963d4e TargetServiceName=null EntryTimestamp=2022-04-26T06:25:10.963Z InvocationID=a69508e4-c76c-4e74-95a4-b248874d29d5 ErrorCode=500 Starting to create Csar for component OntPnf 2022-04-26T06:25:11.929Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "OntPnf" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a80ba61f-597e-414e-82f9-6d34548fc61d" INVARIANT_UUID = "634ff3dc-bab6-4d05-953d-6c8f2a107fa8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "93edbabf-e707-43bc-b852-41cb0129f511.assettoscatemplate" CURR_ARTIFACT_UUID = "93edbabf-e707-43bc-b852-41cb0129f511.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ontpnf-template.yml,0,93edbabf-e707-43bc-b852-41cb0129f511.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7187a394-3105-4001-bace-262cb2963d4e TargetServiceName=null EntryTimestamp=2022-04-26T06:25:10.963Z InvocationID=a69508e4-c76c-4e74-95a4-b248874d29d5 ErrorCode=500 Ended create Csar for component OntPnf 2022-04-26T06:25:11.930Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "OntPnf" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a80ba61f-597e-414e-82f9-6d34548fc61d" INVARIANT_UUID = "634ff3dc-bab6-4d05-953d-6c8f2a107fa8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "93edbabf-e707-43bc-b852-41cb0129f511.assettoscatemplate" CURR_ARTIFACT_UUID = "93edbabf-e707-43bc-b852-41cb0129f511.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ontpnf-template.yml,0,93edbabf-e707-43bc-b852-41cb0129f511.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7187a394-3105-4001-bace-262cb2963d4e TargetServiceName=null EntryTimestamp=2022-04-26T06:25:10.963Z InvocationID=a69508e4-c76c-4e74-95a4-b248874d29d5 ErrorCode=500 lock resource with id 93edbabf-e707-43bc-b852-41cb0129f511 2022-04-26T06:25:12.486Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "OntPnf" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a80ba61f-597e-414e-82f9-6d34548fc61d" INVARIANT_UUID = "634ff3dc-bab6-4d05-953d-6c8f2a107fa8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "93edbabf-e707-43bc-b852-41cb0129f511.assettoscacsar" CURR_ARTIFACT_UUID = "93edbabf-e707-43bc-b852-41cb0129f511.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Ontpnf-csar.csar,0,93edbabf-e707-43bc-b852-41cb0129f511.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1523 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=68f6c539-b222-41bd-821e-9a948883c4a8 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:10.963Z InvocationID=a69508e4-c76c-4e74-95a4-b248874d29d5 ErrorCode=500 LogTimestamp=2022-04-26T06:25:12.486500Z ResponseDesc=Created Exiting. 2022-04-26T06:25:12.487Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:25:12.501Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=500e8b0b-e325-4ac9-9c57-562c9e3df7a2 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:12.501441Z InvocationID=bd0b2d46-08fa-41fe-a247-4aabd0dc414a LogTimestamp=2022-04-26T06:25:12.501528Z Entering 2022-04-26T06:25:12.502Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=500e8b0b-e325-4ac9-9c57-562c9e3df7a2 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:12.501441Z InvocationID=bd0b2d46-08fa-41fe-a247-4aabd0dc414a LogTimestamp=2022-04-26T06:25:12.501528Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:25:12.613Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:12.502Z InvocationID=92161d44-d6a5-4ec1-bc84-f6f14b24eaa7 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:12.614Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:12.502Z InvocationID=92161d44-d6a5-4ec1-bc84-f6f14b24eaa7 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:12.614Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:12.502Z InvocationID=92161d44-d6a5-4ec1-bc84-f6f14b24eaa7 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:12.614Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:12.502Z InvocationID=92161d44-d6a5-4ec1-bc84-f6f14b24eaa7 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:12.623Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:12.502Z InvocationID=92161d44-d6a5-4ec1-bc84-f6f14b24eaa7 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Ponuni 2022-04-26T06:25:12.659Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.233.72.7:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:25:12.679Z [qtp2085002312-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "PonUni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "5c29733e-f54a-4cfd-9c2b-9a7ff4765962" INVARIANT_UUID = "5412bd5d-49ea-4b44-8f6e-31a8783a975e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.PonUni" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5c29733e-f54a-4cfd-9c2b-9a7ff4765962 EntryTimestamp=2022-04-26T06:25:12.502Z InvocationID=92161d44-d6a5-4ec1-bc84-f6f14b24eaa7 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:12.680Z [qtp2085002312-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "PonUni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "5c29733e-f54a-4cfd-9c2b-9a7ff4765962" INVARIANT_UUID = "5412bd5d-49ea-4b44-8f6e-31a8783a975e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.PonUni" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5c29733e-f54a-4cfd-9c2b-9a7ff4765962 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:12.502Z InvocationID=92161d44-d6a5-4ec1-bc84-f6f14b24eaa7 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:12.763Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "PonUni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "5c29733e-f54a-4cfd-9c2b-9a7ff4765962" INVARIANT_UUID = "5412bd5d-49ea-4b44-8f6e-31a8783a975e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.PonUni" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5c29733e-f54a-4cfd-9c2b-9a7ff4765962 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:12.502Z InvocationID=92161d44-d6a5-4ec1-bc84-f6f14b24eaa7 ErrorCode=500 lock resource with id df716dfe-0330-4666-a8b8-ae76bd5e3013 2022-04-26T06:25:12.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6102b2c5-e845-452b-9d57-f48b70da5b0d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:25:12.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6102b2c5-e845-452b-9d57-f48b70da5b0d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@32321e26 2022-04-26T06:25:12.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6102b2c5-e845-452b-9d57-f48b70da5b0d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:25:12.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6102b2c5-e845-452b-9d57-f48b70da5b0d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:25:13.499Z [qtp2085002312-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "PonUni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "5c29733e-f54a-4cfd-9c2b-9a7ff4765962" INVARIANT_UUID = "5412bd5d-49ea-4b44-8f6e-31a8783a975e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "df716dfe-0330-4666-a8b8-ae76bd5e3013.assettoscatemplate" CURR_ARTIFACT_UUID = "df716dfe-0330-4666-a8b8-ae76bd5e3013.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ponuni-template.yml,0,df716dfe-0330-4666-a8b8-ae76bd5e3013.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d4594f03-e109-4f51-bbf9-710d8b951ec8 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:12.502Z InvocationID=92161d44-d6a5-4ec1-bc84-f6f14b24eaa7 ErrorCode=500 Starting to create Csar for component PonUni 2022-04-26T06:25:13.677Z [qtp2085002312-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "PonUni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "5c29733e-f54a-4cfd-9c2b-9a7ff4765962" INVARIANT_UUID = "5412bd5d-49ea-4b44-8f6e-31a8783a975e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "df716dfe-0330-4666-a8b8-ae76bd5e3013.assettoscatemplate" CURR_ARTIFACT_UUID = "df716dfe-0330-4666-a8b8-ae76bd5e3013.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ponuni-template.yml,0,df716dfe-0330-4666-a8b8-ae76bd5e3013.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d4594f03-e109-4f51-bbf9-710d8b951ec8 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:12.502Z InvocationID=92161d44-d6a5-4ec1-bc84-f6f14b24eaa7 ErrorCode=500 Ended create Csar for component PonUni 2022-04-26T06:25:13.677Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "PonUni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "5c29733e-f54a-4cfd-9c2b-9a7ff4765962" INVARIANT_UUID = "5412bd5d-49ea-4b44-8f6e-31a8783a975e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "df716dfe-0330-4666-a8b8-ae76bd5e3013.assettoscatemplate" CURR_ARTIFACT_UUID = "df716dfe-0330-4666-a8b8-ae76bd5e3013.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ponuni-template.yml,0,df716dfe-0330-4666-a8b8-ae76bd5e3013.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d4594f03-e109-4f51-bbf9-710d8b951ec8 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:12.502Z InvocationID=92161d44-d6a5-4ec1-bc84-f6f14b24eaa7 ErrorCode=500 lock resource with id df716dfe-0330-4666-a8b8-ae76bd5e3013 2022-04-26T06:25:13.902Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=6102b2c5-e845-452b-9d57-f48b70da5b0d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:25:13.903Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=6102b2c5-e845-452b-9d57-f48b70da5b0d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:25:14.355Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "PonUni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "5c29733e-f54a-4cfd-9c2b-9a7ff4765962" INVARIANT_UUID = "5412bd5d-49ea-4b44-8f6e-31a8783a975e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "df716dfe-0330-4666-a8b8-ae76bd5e3013.assettoscacsar" CURR_ARTIFACT_UUID = "df716dfe-0330-4666-a8b8-ae76bd5e3013.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Ponuni-csar.csar,0,df716dfe-0330-4666-a8b8-ae76bd5e3013.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1852 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=dcce4fbf-8c6b-43be-b369-f6cb66e5754a TargetServiceName=null EntryTimestamp=2022-04-26T06:25:12.502Z InvocationID=92161d44-d6a5-4ec1-bc84-f6f14b24eaa7 ErrorCode=500 LogTimestamp=2022-04-26T06:25:14.354770Z ResponseDesc=Created Exiting. 2022-04-26T06:25:14.355Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:25:14.368Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5c75b0bf-8412-45b7-b90d-bbd1d9f0fe22 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:14.368073Z InvocationID=cf49d38c-29d9-44ce-9bf7-743c6a7408b9 LogTimestamp=2022-04-26T06:25:14.368120Z Entering 2022-04-26T06:25:14.368Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5c75b0bf-8412-45b7-b90d-bbd1d9f0fe22 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:14.368073Z InvocationID=cf49d38c-29d9-44ce-9bf7-743c6a7408b9 LogTimestamp=2022-04-26T06:25:14.368120Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:25:14.574Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:14.368Z InvocationID=2bad570b-22aa-42c3-97d9-9d08c5d7ceac PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:14.574Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:14.368Z InvocationID=2bad570b-22aa-42c3-97d9-9d08c5d7ceac PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:14.574Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:14.368Z InvocationID=2bad570b-22aa-42c3-97d9-9d08c5d7ceac PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:14.574Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:14.368Z InvocationID=2bad570b-22aa-42c3-97d9-9d08c5d7ceac PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:14.582Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:14.368Z InvocationID=2bad570b-22aa-42c3-97d9-9d08c5d7ceac PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Oltnni 2022-04-26T06:25:14.665Z [qtp2085002312-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "OltNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "5a232a07-9aea-4e99-a323-76f0f325ede2" INVARIANT_UUID = "b5da9653-e5ce-4fa4-ba9b-526e7632f851" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.OltNni" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5a232a07-9aea-4e99-a323-76f0f325ede2 EntryTimestamp=2022-04-26T06:25:14.368Z InvocationID=2bad570b-22aa-42c3-97d9-9d08c5d7ceac PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:14.666Z [qtp2085002312-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "OltNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "5a232a07-9aea-4e99-a323-76f0f325ede2" INVARIANT_UUID = "b5da9653-e5ce-4fa4-ba9b-526e7632f851" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.OltNni" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5a232a07-9aea-4e99-a323-76f0f325ede2 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:14.368Z InvocationID=2bad570b-22aa-42c3-97d9-9d08c5d7ceac ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:14.759Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "OltNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "5a232a07-9aea-4e99-a323-76f0f325ede2" INVARIANT_UUID = "b5da9653-e5ce-4fa4-ba9b-526e7632f851" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.OltNni" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5a232a07-9aea-4e99-a323-76f0f325ede2 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:14.368Z InvocationID=2bad570b-22aa-42c3-97d9-9d08c5d7ceac ErrorCode=500 lock resource with id 253cebbb-86a0-4b62-9080-56fac1379360 2022-04-26T06:25:15.550Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "OltNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "5a232a07-9aea-4e99-a323-76f0f325ede2" INVARIANT_UUID = "b5da9653-e5ce-4fa4-ba9b-526e7632f851" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "253cebbb-86a0-4b62-9080-56fac1379360.assettoscatemplate" CURR_ARTIFACT_UUID = "253cebbb-86a0-4b62-9080-56fac1379360.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Oltnni-template.yml,0,253cebbb-86a0-4b62-9080-56fac1379360.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8ab2c0b2-303f-4c6b-8ed7-d5a029350d07 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:14.368Z InvocationID=2bad570b-22aa-42c3-97d9-9d08c5d7ceac ErrorCode=500 Starting to create Csar for component OltNni 2022-04-26T06:25:15.721Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "OltNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "5a232a07-9aea-4e99-a323-76f0f325ede2" INVARIANT_UUID = "b5da9653-e5ce-4fa4-ba9b-526e7632f851" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "253cebbb-86a0-4b62-9080-56fac1379360.assettoscatemplate" CURR_ARTIFACT_UUID = "253cebbb-86a0-4b62-9080-56fac1379360.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Oltnni-template.yml,0,253cebbb-86a0-4b62-9080-56fac1379360.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8ab2c0b2-303f-4c6b-8ed7-d5a029350d07 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:14.368Z InvocationID=2bad570b-22aa-42c3-97d9-9d08c5d7ceac ErrorCode=500 Ended create Csar for component OltNni 2022-04-26T06:25:15.721Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "OltNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "5a232a07-9aea-4e99-a323-76f0f325ede2" INVARIANT_UUID = "b5da9653-e5ce-4fa4-ba9b-526e7632f851" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "253cebbb-86a0-4b62-9080-56fac1379360.assettoscatemplate" CURR_ARTIFACT_UUID = "253cebbb-86a0-4b62-9080-56fac1379360.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Oltnni-template.yml,0,253cebbb-86a0-4b62-9080-56fac1379360.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8ab2c0b2-303f-4c6b-8ed7-d5a029350d07 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:14.368Z InvocationID=2bad570b-22aa-42c3-97d9-9d08c5d7ceac ErrorCode=500 lock resource with id 253cebbb-86a0-4b62-9080-56fac1379360 2022-04-26T06:25:15.939Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2423b010-e5f7-4aca-9d0f-817aec43fa59 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:25:15.939Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2423b010-e5f7-4aca-9d0f-817aec43fa59 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@43874d2d 2022-04-26T06:25:15.939Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2423b010-e5f7-4aca-9d0f-817aec43fa59 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:25:15.939Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2423b010-e5f7-4aca-9d0f-817aec43fa59 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:25:16.350Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "OltNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "5a232a07-9aea-4e99-a323-76f0f325ede2" INVARIANT_UUID = "b5da9653-e5ce-4fa4-ba9b-526e7632f851" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "253cebbb-86a0-4b62-9080-56fac1379360.assettoscacsar" CURR_ARTIFACT_UUID = "253cebbb-86a0-4b62-9080-56fac1379360.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Oltnni-csar.csar,0,253cebbb-86a0-4b62-9080-56fac1379360.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1982 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=37edbea5-5b2f-4bea-8855-5897775d37f9 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:14.368Z InvocationID=2bad570b-22aa-42c3-97d9-9d08c5d7ceac ErrorCode=500 LogTimestamp=2022-04-26T06:25:16.350617Z ResponseDesc=Created Exiting. 2022-04-26T06:25:16.351Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:25:16.363Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=e0e0135a-9950-403b-a0b8-ac0298ae45e2 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:16.363631Z InvocationID=d1a283ea-e5a3-4a7d-98d4-104099496cc3 LogTimestamp=2022-04-26T06:25:16.363701Z Entering 2022-04-26T06:25:16.364Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=e0e0135a-9950-403b-a0b8-ac0298ae45e2 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:16.363631Z InvocationID=d1a283ea-e5a3-4a7d-98d4-104099496cc3 LogTimestamp=2022-04-26T06:25:16.363701Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:25:16.552Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:16.364Z InvocationID=ad6f1280-112b-43ba-96b0-eb00fa8669fa PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:16.552Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:16.364Z InvocationID=ad6f1280-112b-43ba-96b0-eb00fa8669fa PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:16.552Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:16.364Z InvocationID=ad6f1280-112b-43ba-96b0-eb00fa8669fa PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:16.553Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:16.364Z InvocationID=ad6f1280-112b-43ba-96b0-eb00fa8669fa PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:16.559Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:16.364Z InvocationID=ad6f1280-112b-43ba-96b0-eb00fa8669fa PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Ontnni 2022-04-26T06:25:16.651Z [qtp2085002312-20] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "OntNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "8e280cd5-e92c-4b01-8543-327754150223" INVARIANT_UUID = "8073f6da-bd37-4425-8cc4-abd45268dcb6" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.OntNni" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8e280cd5-e92c-4b01-8543-327754150223 EntryTimestamp=2022-04-26T06:25:16.364Z InvocationID=ad6f1280-112b-43ba-96b0-eb00fa8669fa PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:16.652Z [qtp2085002312-20] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "OntNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "8e280cd5-e92c-4b01-8543-327754150223" INVARIANT_UUID = "8073f6da-bd37-4425-8cc4-abd45268dcb6" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.OntNni" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8e280cd5-e92c-4b01-8543-327754150223 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:16.364Z InvocationID=ad6f1280-112b-43ba-96b0-eb00fa8669fa ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:16.740Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "OntNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "8e280cd5-e92c-4b01-8543-327754150223" INVARIANT_UUID = "8073f6da-bd37-4425-8cc4-abd45268dcb6" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.OntNni" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8e280cd5-e92c-4b01-8543-327754150223 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:16.364Z InvocationID=ad6f1280-112b-43ba-96b0-eb00fa8669fa ErrorCode=500 lock resource with id 69c16d21-da32-46bc-acd1-353d3b7e8de8 2022-04-26T06:25:16.969Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=2423b010-e5f7-4aca-9d0f-817aec43fa59 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:25:16.969Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=2423b010-e5f7-4aca-9d0f-817aec43fa59 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:25:17.354Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "OntNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "8e280cd5-e92c-4b01-8543-327754150223" INVARIANT_UUID = "8073f6da-bd37-4425-8cc4-abd45268dcb6" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "69c16d21-da32-46bc-acd1-353d3b7e8de8.assettoscatemplate" CURR_ARTIFACT_UUID = "69c16d21-da32-46bc-acd1-353d3b7e8de8.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ontnni-template.yml,0,69c16d21-da32-46bc-acd1-353d3b7e8de8.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=df73839d-6abc-4ca3-a141-f3b90def6691 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:16.364Z InvocationID=ad6f1280-112b-43ba-96b0-eb00fa8669fa ErrorCode=500 Starting to create Csar for component OntNni 2022-04-26T06:25:17.507Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "OntNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "8e280cd5-e92c-4b01-8543-327754150223" INVARIANT_UUID = "8073f6da-bd37-4425-8cc4-abd45268dcb6" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "69c16d21-da32-46bc-acd1-353d3b7e8de8.assettoscatemplate" CURR_ARTIFACT_UUID = "69c16d21-da32-46bc-acd1-353d3b7e8de8.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ontnni-template.yml,0,69c16d21-da32-46bc-acd1-353d3b7e8de8.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=df73839d-6abc-4ca3-a141-f3b90def6691 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:16.364Z InvocationID=ad6f1280-112b-43ba-96b0-eb00fa8669fa ErrorCode=500 Ended create Csar for component OntNni 2022-04-26T06:25:17.508Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "OntNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "8e280cd5-e92c-4b01-8543-327754150223" INVARIANT_UUID = "8073f6da-bd37-4425-8cc4-abd45268dcb6" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "69c16d21-da32-46bc-acd1-353d3b7e8de8.assettoscatemplate" CURR_ARTIFACT_UUID = "69c16d21-da32-46bc-acd1-353d3b7e8de8.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ontnni-template.yml,0,69c16d21-da32-46bc-acd1-353d3b7e8de8.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=df73839d-6abc-4ca3-a141-f3b90def6691 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:16.364Z InvocationID=ad6f1280-112b-43ba-96b0-eb00fa8669fa ErrorCode=500 lock resource with id 69c16d21-da32-46bc-acd1-353d3b7e8de8 2022-04-26T06:25:18.058Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "OntNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "8e280cd5-e92c-4b01-8543-327754150223" INVARIANT_UUID = "8073f6da-bd37-4425-8cc4-abd45268dcb6" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "69c16d21-da32-46bc-acd1-353d3b7e8de8.assettoscacsar" CURR_ARTIFACT_UUID = "69c16d21-da32-46bc-acd1-353d3b7e8de8.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Ontnni-csar.csar,0,69c16d21-da32-46bc-acd1-353d3b7e8de8.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1693 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8135e87d-3f2b-410c-b708-9fdbc40adc75 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:16.364Z InvocationID=ad6f1280-112b-43ba-96b0-eb00fa8669fa ErrorCode=500 LogTimestamp=2022-04-26T06:25:18.057670Z ResponseDesc=Created Exiting. 2022-04-26T06:25:18.058Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:25:18.071Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f31a53f4-a886-47e4-ba0c-314a05601c20 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:18.070732Z InvocationID=cdd8ecd9-e279-4fb5-8743-7b277184509e LogTimestamp=2022-04-26T06:25:18.070794Z Entering 2022-04-26T06:25:18.071Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f31a53f4-a886-47e4-ba0c-314a05601c20 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:18.070732Z InvocationID=cdd8ecd9-e279-4fb5-8743-7b277184509e LogTimestamp=2022-04-26T06:25:18.070794Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:25:18.148Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:18.071Z InvocationID=9f09fd21-9fa6-4773-b695-b2b316348c86 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:18.149Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:18.071Z InvocationID=9f09fd21-9fa6-4773-b695-b2b316348c86 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:18.149Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:18.071Z InvocationID=9f09fd21-9fa6-4773-b695-b2b316348c86 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:18.149Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:18.071Z InvocationID=9f09fd21-9fa6-4773-b695-b2b316348c86 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:18.152Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:18.071Z InvocationID=9f09fd21-9fa6-4773-b695-b2b316348c86 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Sap 2022-04-26T06:25:18.160Z [cluster2-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.233.72.7:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:25:18.161Z [cluster2-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.233.72.7:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:25:18.162Z [cluster2-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.233.72.7:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:25:18.192Z [qtp2085002312-15] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Sap" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "c349988f-3ee5-4afc-8d12-0e857ba577e8" INVARIANT_UUID = "a305a2d5-90cf-4bfd-900c-d38ccb1683a1" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Sap" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c349988f-3ee5-4afc-8d12-0e857ba577e8 EntryTimestamp=2022-04-26T06:25:18.071Z InvocationID=9f09fd21-9fa6-4773-b695-b2b316348c86 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:18.193Z [qtp2085002312-15] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Sap" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "c349988f-3ee5-4afc-8d12-0e857ba577e8" INVARIANT_UUID = "a305a2d5-90cf-4bfd-900c-d38ccb1683a1" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Sap" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c349988f-3ee5-4afc-8d12-0e857ba577e8 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:18.071Z InvocationID=9f09fd21-9fa6-4773-b695-b2b316348c86 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:18.235Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Sap" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "c349988f-3ee5-4afc-8d12-0e857ba577e8" INVARIANT_UUID = "a305a2d5-90cf-4bfd-900c-d38ccb1683a1" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Sap" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c349988f-3ee5-4afc-8d12-0e857ba577e8 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:18.071Z InvocationID=9f09fd21-9fa6-4773-b695-b2b316348c86 ErrorCode=500 lock resource with id c5e9e015-621a-4255-ba02-6db4a129e5ab 2022-04-26T06:25:18.249Z [cluster2-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.233.72.7:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:25:18.250Z [cluster2-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.233.72.7:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:25:18.455Z [cluster1-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:25:18.456Z [cluster1-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 2000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:25:18.457Z [cluster2-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.233.72.7:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:25:18.458Z [JanusGraph Cluster-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.233.72.7:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:25:18.842Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=104d0244-a5cb-48d7-9783-30ed5ddb4e24 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:25:18.842Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=104d0244-a5cb-48d7-9783-30ed5ddb4e24 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@51c4ccd0 2022-04-26T06:25:18.842Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=104d0244-a5cb-48d7-9783-30ed5ddb4e24 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:25:18.842Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=104d0244-a5cb-48d7-9783-30ed5ddb4e24 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:25:18.938Z [qtp2085002312-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Sap" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "c349988f-3ee5-4afc-8d12-0e857ba577e8" INVARIANT_UUID = "a305a2d5-90cf-4bfd-900c-d38ccb1683a1" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c5e9e015-621a-4255-ba02-6db4a129e5ab.assettoscatemplate" CURR_ARTIFACT_UUID = "c5e9e015-621a-4255-ba02-6db4a129e5ab.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Sap-template.yml,0,c5e9e015-621a-4255-ba02-6db4a129e5ab.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d24eb112-374d-46a6-82e9-358736e6762d TargetServiceName=null EntryTimestamp=2022-04-26T06:25:18.071Z InvocationID=9f09fd21-9fa6-4773-b695-b2b316348c86 ErrorCode=500 Starting to create Csar for component Sap 2022-04-26T06:25:19.139Z [qtp2085002312-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Sap" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "c349988f-3ee5-4afc-8d12-0e857ba577e8" INVARIANT_UUID = "a305a2d5-90cf-4bfd-900c-d38ccb1683a1" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c5e9e015-621a-4255-ba02-6db4a129e5ab.assettoscatemplate" CURR_ARTIFACT_UUID = "c5e9e015-621a-4255-ba02-6db4a129e5ab.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Sap-template.yml,0,c5e9e015-621a-4255-ba02-6db4a129e5ab.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d24eb112-374d-46a6-82e9-358736e6762d TargetServiceName=null EntryTimestamp=2022-04-26T06:25:18.071Z InvocationID=9f09fd21-9fa6-4773-b695-b2b316348c86 ErrorCode=500 Ended create Csar for component Sap 2022-04-26T06:25:19.140Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Sap" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "c349988f-3ee5-4afc-8d12-0e857ba577e8" INVARIANT_UUID = "a305a2d5-90cf-4bfd-900c-d38ccb1683a1" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c5e9e015-621a-4255-ba02-6db4a129e5ab.assettoscatemplate" CURR_ARTIFACT_UUID = "c5e9e015-621a-4255-ba02-6db4a129e5ab.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Sap-template.yml,0,c5e9e015-621a-4255-ba02-6db4a129e5ab.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d24eb112-374d-46a6-82e9-358736e6762d TargetServiceName=null EntryTimestamp=2022-04-26T06:25:18.071Z InvocationID=9f09fd21-9fa6-4773-b695-b2b316348c86 ErrorCode=500 lock resource with id c5e9e015-621a-4255-ba02-6db4a129e5ab 2022-04-26T06:25:19.760Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Sap" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "c349988f-3ee5-4afc-8d12-0e857ba577e8" INVARIANT_UUID = "a305a2d5-90cf-4bfd-900c-d38ccb1683a1" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c5e9e015-621a-4255-ba02-6db4a129e5ab.assettoscacsar" CURR_ARTIFACT_UUID = "c5e9e015-621a-4255-ba02-6db4a129e5ab.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Sap-csar.csar,0,c5e9e015-621a-4255-ba02-6db4a129e5ab.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1688 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6e281583-38b6-49ff-8d07-14033614c269 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:18.071Z InvocationID=9f09fd21-9fa6-4773-b695-b2b316348c86 ErrorCode=500 LogTimestamp=2022-04-26T06:25:19.759771Z ResponseDesc=Created Exiting. 2022-04-26T06:25:19.760Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:25:19.842Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=2fc17659-e631-49fc-b09b-06b84486b69d InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:19.841888Z InvocationID=d8d7b5ff-5e88-4052-a9fa-c48f9e7ae321 LogTimestamp=2022-04-26T06:25:19.841955Z Entering 2022-04-26T06:25:19.842Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=2fc17659-e631-49fc-b09b-06b84486b69d InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:19.841888Z InvocationID=d8d7b5ff-5e88-4052-a9fa-c48f9e7ae321 LogTimestamp=2022-04-26T06:25:19.841955Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:25:19.885Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=104d0244-a5cb-48d7-9783-30ed5ddb4e24 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:25:19.885Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=104d0244-a5cb-48d7-9783-30ed5ddb4e24 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:25:20.141Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:19.842Z InvocationID=27b6a6a3-f892-4d7e-9fd9-2dd4ffa81a98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:20.141Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:19.842Z InvocationID=27b6a6a3-f892-4d7e-9fd9-2dd4ffa81a98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:20.141Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:19.842Z InvocationID=27b6a6a3-f892-4d7e-9fd9-2dd4ffa81a98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:20.142Z [qtp2085002312-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:19.842Z InvocationID=27b6a6a3-f892-4d7e-9fd9-2dd4ffa81a98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:20.156Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:19.842Z InvocationID=27b6a6a3-f892-4d7e-9fd9-2dd4ffa81a98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Nfppositionelement 2022-04-26T06:25:20.250Z [qtp2085002312-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NfpPositionElement" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "4ec1b7a8-1bfb-4836-bc01-23cf305c77d6" INVARIANT_UUID = "e2493a01-3e0c-443a-b279-581acd825cf3" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NfpPositionElement" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4ec1b7a8-1bfb-4836-bc01-23cf305c77d6 EntryTimestamp=2022-04-26T06:25:19.842Z InvocationID=27b6a6a3-f892-4d7e-9fd9-2dd4ffa81a98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:20.251Z [qtp2085002312-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NfpPositionElement" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "4ec1b7a8-1bfb-4836-bc01-23cf305c77d6" INVARIANT_UUID = "e2493a01-3e0c-443a-b279-581acd825cf3" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NfpPositionElement" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4ec1b7a8-1bfb-4836-bc01-23cf305c77d6 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:19.842Z InvocationID=27b6a6a3-f892-4d7e-9fd9-2dd4ffa81a98 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:20.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:25:20.370Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NfpPositionElement" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "4ec1b7a8-1bfb-4836-bc01-23cf305c77d6" INVARIANT_UUID = "e2493a01-3e0c-443a-b279-581acd825cf3" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NfpPositionElement" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4ec1b7a8-1bfb-4836-bc01-23cf305c77d6 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:19.842Z InvocationID=27b6a6a3-f892-4d7e-9fd9-2dd4ffa81a98 ErrorCode=500 lock resource with id b07e608a-844e-488f-adee-1a54f1fdfb06 2022-04-26T06:25:20.385Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:25:21.442Z [qtp2085002312-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NfpPositionElement" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "4ec1b7a8-1bfb-4836-bc01-23cf305c77d6" INVARIANT_UUID = "e2493a01-3e0c-443a-b279-581acd825cf3" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b07e608a-844e-488f-adee-1a54f1fdfb06.assettoscatemplate" CURR_ARTIFACT_UUID = "b07e608a-844e-488f-adee-1a54f1fdfb06.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nfppositionelement-template.yml,0,b07e608a-844e-488f-adee-1a54f1fdfb06.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bba9fec2-1e6e-4f0b-9b0c-f0232be77479 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:19.842Z InvocationID=27b6a6a3-f892-4d7e-9fd9-2dd4ffa81a98 ErrorCode=500 Starting to create Csar for component NfpPositionElement 2022-04-26T06:25:21.630Z [qtp2085002312-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NfpPositionElement" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "4ec1b7a8-1bfb-4836-bc01-23cf305c77d6" INVARIANT_UUID = "e2493a01-3e0c-443a-b279-581acd825cf3" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b07e608a-844e-488f-adee-1a54f1fdfb06.assettoscatemplate" CURR_ARTIFACT_UUID = "b07e608a-844e-488f-adee-1a54f1fdfb06.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nfppositionelement-template.yml,0,b07e608a-844e-488f-adee-1a54f1fdfb06.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bba9fec2-1e6e-4f0b-9b0c-f0232be77479 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:19.842Z InvocationID=27b6a6a3-f892-4d7e-9fd9-2dd4ffa81a98 ErrorCode=500 Ended create Csar for component NfpPositionElement 2022-04-26T06:25:21.631Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NfpPositionElement" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "4ec1b7a8-1bfb-4836-bc01-23cf305c77d6" INVARIANT_UUID = "e2493a01-3e0c-443a-b279-581acd825cf3" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b07e608a-844e-488f-adee-1a54f1fdfb06.assettoscatemplate" CURR_ARTIFACT_UUID = "b07e608a-844e-488f-adee-1a54f1fdfb06.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nfppositionelement-template.yml,0,b07e608a-844e-488f-adee-1a54f1fdfb06.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bba9fec2-1e6e-4f0b-9b0c-f0232be77479 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:19.842Z InvocationID=27b6a6a3-f892-4d7e-9fd9-2dd4ffa81a98 ErrorCode=500 lock resource with id b07e608a-844e-488f-adee-1a54f1fdfb06 2022-04-26T06:25:22.444Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NfpPositionElement" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "4ec1b7a8-1bfb-4836-bc01-23cf305c77d6" INVARIANT_UUID = "e2493a01-3e0c-443a-b279-581acd825cf3" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b07e608a-844e-488f-adee-1a54f1fdfb06.assettoscacsar" CURR_ARTIFACT_UUID = "b07e608a-844e-488f-adee-1a54f1fdfb06.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Nfppositionelement-csar.csar,0,b07e608a-844e-488f-adee-1a54f1fdfb06.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=2601 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9fad8563-f201-4235-bdcc-f7683ca9f24f TargetServiceName=null EntryTimestamp=2022-04-26T06:25:19.842Z InvocationID=27b6a6a3-f892-4d7e-9fd9-2dd4ffa81a98 ErrorCode=500 LogTimestamp=2022-04-26T06:25:22.443765Z ResponseDesc=Created Exiting. 2022-04-26T06:25:22.444Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:25:22.462Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=09cff2f0-aee2-412e-a3db-66410a6e62ec InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:22.461838Z InvocationID=df6de0b9-2c47-4051-8437-35edac5db91a LogTimestamp=2022-04-26T06:25:22.461911Z Entering 2022-04-26T06:25:22.462Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=09cff2f0-aee2-412e-a3db-66410a6e62ec InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:22.461838Z InvocationID=df6de0b9-2c47-4051-8437-35edac5db91a LogTimestamp=2022-04-26T06:25:22.461911Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:25:22.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=65aa88a5-e4da-4a35-ba6a-be794ceb150e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:25:22.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=65aa88a5-e4da-4a35-ba6a-be794ceb150e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@708c2d12 2022-04-26T06:25:22.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=65aa88a5-e4da-4a35-ba6a-be794ceb150e PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:25:22.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=65aa88a5-e4da-4a35-ba6a-be794ceb150e PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:25:23.044Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:22.462Z InvocationID=7d2c3251-bf63-4e46-bd15-0c43b8e52399 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:23.045Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:22.462Z InvocationID=7d2c3251-bf63-4e46-bd15-0c43b8e52399 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:23.045Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:22.462Z InvocationID=7d2c3251-bf63-4e46-bd15-0c43b8e52399 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:23.045Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:22.462Z InvocationID=7d2c3251-bf63-4e46-bd15-0c43b8e52399 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:23.045Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:22.462Z InvocationID=7d2c3251-bf63-4e46-bd15-0c43b8e52399 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Nfpposition 2022-04-26T06:25:23.243Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NfpPosition" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "efb09199-0e48-4d73-8a63-417f35f2f199" INVARIANT_UUID = "e112a0df-c8f2-4069-af2e-0227c86f53f9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NfpPosition" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=efb09199-0e48-4d73-8a63-417f35f2f199 EntryTimestamp=2022-04-26T06:25:22.462Z InvocationID=7d2c3251-bf63-4e46-bd15-0c43b8e52399 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:23.244Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NfpPosition" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "efb09199-0e48-4d73-8a63-417f35f2f199" INVARIANT_UUID = "e112a0df-c8f2-4069-af2e-0227c86f53f9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NfpPosition" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=efb09199-0e48-4d73-8a63-417f35f2f199 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:22.462Z InvocationID=7d2c3251-bf63-4e46-bd15-0c43b8e52399 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:23.457Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NfpPosition" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "efb09199-0e48-4d73-8a63-417f35f2f199" INVARIANT_UUID = "e112a0df-c8f2-4069-af2e-0227c86f53f9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NfpPosition" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=efb09199-0e48-4d73-8a63-417f35f2f199 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:22.462Z InvocationID=7d2c3251-bf63-4e46-bd15-0c43b8e52399 ErrorCode=500 lock resource with id 5b1b8a97-38ca-47fa-ab6f-05e8f4ada971 2022-04-26T06:25:23.656Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=65aa88a5-e4da-4a35-ba6a-be794ceb150e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:25:23.657Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=65aa88a5-e4da-4a35-ba6a-be794ceb150e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:25:24.640Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NfpPosition" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "efb09199-0e48-4d73-8a63-417f35f2f199" INVARIANT_UUID = "e112a0df-c8f2-4069-af2e-0227c86f53f9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5b1b8a97-38ca-47fa-ab6f-05e8f4ada971.assettoscatemplate" CURR_ARTIFACT_UUID = "5b1b8a97-38ca-47fa-ab6f-05e8f4ada971.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nfpposition-template.yml,0,5b1b8a97-38ca-47fa-ab6f-05e8f4ada971.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5621724b-af99-4daf-b543-5caba2fb1824 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:22.462Z InvocationID=7d2c3251-bf63-4e46-bd15-0c43b8e52399 ErrorCode=500 Starting to create Csar for component NfpPosition 2022-04-26T06:25:24.796Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NfpPosition" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "efb09199-0e48-4d73-8a63-417f35f2f199" INVARIANT_UUID = "e112a0df-c8f2-4069-af2e-0227c86f53f9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5b1b8a97-38ca-47fa-ab6f-05e8f4ada971.assettoscatemplate" CURR_ARTIFACT_UUID = "5b1b8a97-38ca-47fa-ab6f-05e8f4ada971.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nfpposition-template.yml,0,5b1b8a97-38ca-47fa-ab6f-05e8f4ada971.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5621724b-af99-4daf-b543-5caba2fb1824 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:22.462Z InvocationID=7d2c3251-bf63-4e46-bd15-0c43b8e52399 ErrorCode=500 Ended create Csar for component NfpPosition 2022-04-26T06:25:24.797Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NfpPosition" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "efb09199-0e48-4d73-8a63-417f35f2f199" INVARIANT_UUID = "e112a0df-c8f2-4069-af2e-0227c86f53f9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5b1b8a97-38ca-47fa-ab6f-05e8f4ada971.assettoscatemplate" CURR_ARTIFACT_UUID = "5b1b8a97-38ca-47fa-ab6f-05e8f4ada971.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nfpposition-template.yml,0,5b1b8a97-38ca-47fa-ab6f-05e8f4ada971.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5621724b-af99-4daf-b543-5caba2fb1824 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:22.462Z InvocationID=7d2c3251-bf63-4e46-bd15-0c43b8e52399 ErrorCode=500 lock resource with id 5b1b8a97-38ca-47fa-ab6f-05e8f4ada971 2022-04-26T06:25:25.656Z [JanusGraph Cluster-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:25:25.656Z [cluster1-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:25:25.657Z [JanusGraph Cluster-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 2000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:25:25.658Z [cluster1-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 4000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:25:25.658Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NfpPosition" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "efb09199-0e48-4d73-8a63-417f35f2f199" INVARIANT_UUID = "e112a0df-c8f2-4069-af2e-0227c86f53f9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5b1b8a97-38ca-47fa-ab6f-05e8f4ada971.assettoscacsar" CURR_ARTIFACT_UUID = "5b1b8a97-38ca-47fa-ab6f-05e8f4ada971.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Nfpposition-csar.csar,0,5b1b8a97-38ca-47fa-ab6f-05e8f4ada971.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=3193 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=55ea7d61-4790-4b14-b22b-75bcdeb8760e TargetServiceName=null EntryTimestamp=2022-04-26T06:25:22.462Z InvocationID=7d2c3251-bf63-4e46-bd15-0c43b8e52399 ErrorCode=500 LogTimestamp=2022-04-26T06:25:25.655830Z ResponseDesc=Created Exiting. 2022-04-26T06:25:25.659Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:25:25.741Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ed498a7-b5dd-44a7-9d23-6811d5e61de4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:25:25.741Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ed498a7-b5dd-44a7-9d23-6811d5e61de4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5d5b5fbd 2022-04-26T06:25:25.741Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ed498a7-b5dd-44a7-9d23-6811d5e61de4 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:25:25.741Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=9c77e3c6-c7c5-4d0c-b965-fc6e87d05c9b InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:25.741471Z InvocationID=d10b227b-89a0-49cd-b43b-68c0cb47c86e LogTimestamp=2022-04-26T06:25:25.741539Z Entering 2022-04-26T06:25:25.741Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ed498a7-b5dd-44a7-9d23-6811d5e61de4 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:25:25.741Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=9c77e3c6-c7c5-4d0c-b965-fc6e87d05c9b InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:25.741471Z InvocationID=d10b227b-89a0-49cd-b43b-68c0cb47c86e LogTimestamp=2022-04-26T06:25:25.741539Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:25:26.157Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:25.742Z InvocationID=482c4f28-7809-43d1-bd8c-f9c24f3e5266 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:26.157Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:25.742Z InvocationID=482c4f28-7809-43d1-bd8c-f9c24f3e5266 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:26.157Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:25.742Z InvocationID=482c4f28-7809-43d1-bd8c-f9c24f3e5266 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:26.157Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:25.742Z InvocationID=482c4f28-7809-43d1-bd8c-f9c24f3e5266 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:26.238Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:25.742Z InvocationID=482c4f28-7809-43d1-bd8c-f9c24f3e5266 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Nfp 2022-04-26T06:25:26.441Z [qtp2085002312-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NFP" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "6f3a0f7b-a142-44ca-8a37-ffc66ec687e8" INVARIANT_UUID = "5ec02c58-5286-4ce4-87a0-dbafc2cd40ed" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NFP" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6f3a0f7b-a142-44ca-8a37-ffc66ec687e8 EntryTimestamp=2022-04-26T06:25:25.742Z InvocationID=482c4f28-7809-43d1-bd8c-f9c24f3e5266 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:26.442Z [qtp2085002312-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NFP" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "6f3a0f7b-a142-44ca-8a37-ffc66ec687e8" INVARIANT_UUID = "5ec02c58-5286-4ce4-87a0-dbafc2cd40ed" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NFP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6f3a0f7b-a142-44ca-8a37-ffc66ec687e8 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:25.742Z InvocationID=482c4f28-7809-43d1-bd8c-f9c24f3e5266 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:26.657Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NFP" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "6f3a0f7b-a142-44ca-8a37-ffc66ec687e8" INVARIANT_UUID = "5ec02c58-5286-4ce4-87a0-dbafc2cd40ed" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NFP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6f3a0f7b-a142-44ca-8a37-ffc66ec687e8 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:25.742Z InvocationID=482c4f28-7809-43d1-bd8c-f9c24f3e5266 ErrorCode=500 lock resource with id 9d9fa5f6-c6f7-4b38-a516-4eff0518d64f 2022-04-26T06:25:26.763Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=6ed498a7-b5dd-44a7-9d23-6811d5e61de4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:25:26.764Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=6ed498a7-b5dd-44a7-9d23-6811d5e61de4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:25:27.745Z [qtp2085002312-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NFP" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "6f3a0f7b-a142-44ca-8a37-ffc66ec687e8" INVARIANT_UUID = "5ec02c58-5286-4ce4-87a0-dbafc2cd40ed" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "9d9fa5f6-c6f7-4b38-a516-4eff0518d64f.assettoscatemplate" CURR_ARTIFACT_UUID = "9d9fa5f6-c6f7-4b38-a516-4eff0518d64f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nfp-template.yml,0,9d9fa5f6-c6f7-4b38-a516-4eff0518d64f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ead97ad6-b0c4-4471-b7ac-94845f18cc5c TargetServiceName=null EntryTimestamp=2022-04-26T06:25:25.742Z InvocationID=482c4f28-7809-43d1-bd8c-f9c24f3e5266 ErrorCode=500 Starting to create Csar for component NFP 2022-04-26T06:25:27.918Z [qtp2085002312-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NFP" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "6f3a0f7b-a142-44ca-8a37-ffc66ec687e8" INVARIANT_UUID = "5ec02c58-5286-4ce4-87a0-dbafc2cd40ed" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "9d9fa5f6-c6f7-4b38-a516-4eff0518d64f.assettoscatemplate" CURR_ARTIFACT_UUID = "9d9fa5f6-c6f7-4b38-a516-4eff0518d64f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nfp-template.yml,0,9d9fa5f6-c6f7-4b38-a516-4eff0518d64f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ead97ad6-b0c4-4471-b7ac-94845f18cc5c TargetServiceName=null EntryTimestamp=2022-04-26T06:25:25.742Z InvocationID=482c4f28-7809-43d1-bd8c-f9c24f3e5266 ErrorCode=500 Ended create Csar for component NFP 2022-04-26T06:25:27.919Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NFP" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "6f3a0f7b-a142-44ca-8a37-ffc66ec687e8" INVARIANT_UUID = "5ec02c58-5286-4ce4-87a0-dbafc2cd40ed" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "9d9fa5f6-c6f7-4b38-a516-4eff0518d64f.assettoscatemplate" CURR_ARTIFACT_UUID = "9d9fa5f6-c6f7-4b38-a516-4eff0518d64f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nfp-template.yml,0,9d9fa5f6-c6f7-4b38-a516-4eff0518d64f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ead97ad6-b0c4-4471-b7ac-94845f18cc5c TargetServiceName=null EntryTimestamp=2022-04-26T06:25:25.742Z InvocationID=482c4f28-7809-43d1-bd8c-f9c24f3e5266 ErrorCode=500 lock resource with id 9d9fa5f6-c6f7-4b38-a516-4eff0518d64f 2022-04-26T06:25:28.342Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a6f897f-0c4b-4f35-8372-060216b0ce3c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:25:28.342Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a6f897f-0c4b-4f35-8372-060216b0ce3c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1ada914a 2022-04-26T06:25:28.342Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a6f897f-0c4b-4f35-8372-060216b0ce3c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:25:28.342Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a6f897f-0c4b-4f35-8372-060216b0ce3c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:25:28.746Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NFP" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "6f3a0f7b-a142-44ca-8a37-ffc66ec687e8" INVARIANT_UUID = "5ec02c58-5286-4ce4-87a0-dbafc2cd40ed" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "9d9fa5f6-c6f7-4b38-a516-4eff0518d64f.assettoscacsar" CURR_ARTIFACT_UUID = "9d9fa5f6-c6f7-4b38-a516-4eff0518d64f.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Nfp-csar.csar,0,9d9fa5f6-c6f7-4b38-a516-4eff0518d64f.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=3004 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bb3a7ee7-4b1b-4307-bfba-2a4612aa4887 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:25.742Z InvocationID=482c4f28-7809-43d1-bd8c-f9c24f3e5266 ErrorCode=500 LogTimestamp=2022-04-26T06:25:28.746527Z ResponseDesc=Created Exiting. 2022-04-26T06:25:28.747Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:25:28.842Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=65c68ff7-d6b7-4a2c-b1d9-7f745aad1cb4 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:28.842252Z InvocationID=8c7cc280-f03e-4cd7-8cfe-409eccff4d0e LogTimestamp=2022-04-26T06:25:28.842340Z Entering 2022-04-26T06:25:28.842Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=65c68ff7-d6b7-4a2c-b1d9-7f745aad1cb4 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:28.842252Z InvocationID=8c7cc280-f03e-4cd7-8cfe-409eccff4d0e LogTimestamp=2022-04-26T06:25:28.842340Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:25:29.385Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=5a6f897f-0c4b-4f35-8372-060216b0ce3c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:25:29.385Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=5a6f897f-0c4b-4f35-8372-060216b0ce3c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:25:29.547Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:28.843Z InvocationID=559d439c-94ca-4bcb-873e-bec82efbb64c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:29.547Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:28.843Z InvocationID=559d439c-94ca-4bcb-873e-bec82efbb64c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:29.547Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:28.843Z InvocationID=559d439c-94ca-4bcb-873e-bec82efbb64c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:29.548Z [qtp2085002312-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:28.843Z InvocationID=559d439c-94ca-4bcb-873e-bec82efbb64c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:29.646Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:28.843Z InvocationID=559d439c-94ca-4bcb-873e-bec82efbb64c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name Forwarding 2022-04-26T06:25:30.329Z [qtp2085002312-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Forwarding" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "ff6bb775-ae9e-4f6d-8c02-6a588ca65b04" INVARIANT_UUID = "d1058a36-99be-4ff0-8cb4-a9e390375c1e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Forwarding" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ff6bb775-ae9e-4f6d-8c02-6a588ca65b04 EntryTimestamp=2022-04-26T06:25:28.843Z InvocationID=559d439c-94ca-4bcb-873e-bec82efbb64c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:30.330Z [qtp2085002312-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Forwarding" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "ff6bb775-ae9e-4f6d-8c02-6a588ca65b04" INVARIANT_UUID = "d1058a36-99be-4ff0-8cb4-a9e390375c1e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Forwarding" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ff6bb775-ae9e-4f6d-8c02-6a588ca65b04 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:28.843Z InvocationID=559d439c-94ca-4bcb-873e-bec82efbb64c ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:30.950Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Forwarding" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "ff6bb775-ae9e-4f6d-8c02-6a588ca65b04" INVARIANT_UUID = "d1058a36-99be-4ff0-8cb4-a9e390375c1e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Forwarding" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ff6bb775-ae9e-4f6d-8c02-6a588ca65b04 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:28.843Z InvocationID=559d439c-94ca-4bcb-873e-bec82efbb64c ErrorCode=500 lock resource with id 8147a0ea-a917-471f-bc24-ff0180a8b43c 2022-04-26T06:25:31.641Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bc823c2-ac2c-472f-804a-1a5f5636acc8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:25:31.641Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bc823c2-ac2c-472f-804a-1a5f5636acc8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@428992ea 2022-04-26T06:25:31.641Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bc823c2-ac2c-472f-804a-1a5f5636acc8 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:25:31.641Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bc823c2-ac2c-472f-804a-1a5f5636acc8 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:25:32.653Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3bc823c2-ac2c-472f-804a-1a5f5636acc8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:25:32.653Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3bc823c2-ac2c-472f-804a-1a5f5636acc8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:25:33.330Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Forwarding" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "ff6bb775-ae9e-4f6d-8c02-6a588ca65b04" INVARIANT_UUID = "d1058a36-99be-4ff0-8cb4-a9e390375c1e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8147a0ea-a917-471f-bc24-ff0180a8b43c.assettoscatemplate" CURR_ARTIFACT_UUID = "8147a0ea-a917-471f-bc24-ff0180a8b43c.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Forwarding-template.yml,0,8147a0ea-a917-471f-bc24-ff0180a8b43c.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c5f7cc19-c135-41a0-aa40-4f8df996dc23 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:28.843Z InvocationID=559d439c-94ca-4bcb-873e-bec82efbb64c ErrorCode=500 Starting to create Csar for component Forwarding 2022-04-26T06:25:33.525Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Forwarding" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "ff6bb775-ae9e-4f6d-8c02-6a588ca65b04" INVARIANT_UUID = "d1058a36-99be-4ff0-8cb4-a9e390375c1e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8147a0ea-a917-471f-bc24-ff0180a8b43c.assettoscatemplate" CURR_ARTIFACT_UUID = "8147a0ea-a917-471f-bc24-ff0180a8b43c.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Forwarding-template.yml,0,8147a0ea-a917-471f-bc24-ff0180a8b43c.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c5f7cc19-c135-41a0-aa40-4f8df996dc23 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:28.843Z InvocationID=559d439c-94ca-4bcb-873e-bec82efbb64c ErrorCode=500 Ended create Csar for component Forwarding 2022-04-26T06:25:33.526Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Forwarding" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "ff6bb775-ae9e-4f6d-8c02-6a588ca65b04" INVARIANT_UUID = "d1058a36-99be-4ff0-8cb4-a9e390375c1e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8147a0ea-a917-471f-bc24-ff0180a8b43c.assettoscatemplate" CURR_ARTIFACT_UUID = "8147a0ea-a917-471f-bc24-ff0180a8b43c.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Forwarding-template.yml,0,8147a0ea-a917-471f-bc24-ff0180a8b43c.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c5f7cc19-c135-41a0-aa40-4f8df996dc23 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:28.843Z InvocationID=559d439c-94ca-4bcb-873e-bec82efbb64c ErrorCode=500 lock resource with id 8147a0ea-a917-471f-bc24-ff0180a8b43c 2022-04-26T06:25:34.545Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Forwarding" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "ff6bb775-ae9e-4f6d-8c02-6a588ca65b04" INVARIANT_UUID = "d1058a36-99be-4ff0-8cb4-a9e390375c1e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8147a0ea-a917-471f-bc24-ff0180a8b43c.assettoscacsar" CURR_ARTIFACT_UUID = "8147a0ea-a917-471f-bc24-ff0180a8b43c.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Forwarding-csar.csar,0,8147a0ea-a917-471f-bc24-ff0180a8b43c.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5702 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=66e795e9-b54f-4be4-a163-cce06efa70bf TargetServiceName=null EntryTimestamp=2022-04-26T06:25:28.843Z InvocationID=559d439c-94ca-4bcb-873e-bec82efbb64c ErrorCode=500 LogTimestamp=2022-04-26T06:25:34.545239Z ResponseDesc=Created Exiting. 2022-04-26T06:25:34.546Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:25:34.639Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5709471a-f2b7-4425-88af-b2ffa4ebed70 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:34.638906Z InvocationID=89190a41-c55c-49f9-baee-a224837ada9a LogTimestamp=2022-04-26T06:25:34.638983Z Entering 2022-04-26T06:25:34.639Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5709471a-f2b7-4425-88af-b2ffa4ebed70 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:34.638906Z InvocationID=89190a41-c55c-49f9-baee-a224837ada9a LogTimestamp=2022-04-26T06:25:34.638983Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:25:34.642Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89fc0667-31b9-4f75-b5e9-61546ba65b3a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:25:34.642Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89fc0667-31b9-4f75-b5e9-61546ba65b3a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7901a78a 2022-04-26T06:25:34.642Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89fc0667-31b9-4f75-b5e9-61546ba65b3a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:25:34.642Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89fc0667-31b9-4f75-b5e9-61546ba65b3a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:25:34.656Z [JanusGraph Cluster-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:25:34.657Z [JanusGraph Cluster-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 4000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:25:34.659Z [cluster1-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:25:34.659Z [cluster1-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 8000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:25:35.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:25:35.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:25:35.689Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=89fc0667-31b9-4f75-b5e9-61546ba65b3a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:25:35.689Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=89fc0667-31b9-4f75-b5e9-61546ba65b3a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:25:35.754Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:34.639Z InvocationID=9b2326fd-55bb-4502-b158-3d7e9b0ffd1b PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id c439ea4c-eb18-4f10-98c1-37336155dd48 2022-04-26T06:25:35.932Z [qtp2085002312-14] INFO o.o.s.b.c.l.LifecycleBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:34.639Z InvocationID=9b2326fd-55bb-4502-b158-3d7e9b0ffd1b PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb owner of resource c439ea4c-eb18-4f10-98c1-37336155dd48 is jh0003 2022-04-26T06:25:36.430Z [qtp2085002312-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "4a83a391-0841-425b-b01b-bae244256007" INVARIANT_UUID = "434d09ad-ba73-49db-93e0-4fd09ccbf832" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:34.639Z InvocationID=9b2326fd-55bb-4502-b158-3d7e9b0ffd1b PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:36.431Z [qtp2085002312-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "4a83a391-0841-425b-b01b-bae244256007" INVARIANT_UUID = "434d09ad-ba73-49db-93e0-4fd09ccbf832" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:25:34.639Z InvocationID=9b2326fd-55bb-4502-b158-3d7e9b0ffd1b ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:37.835Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e5c2ece-a4a1-464b-adba-cb217aa78780 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:25:37.835Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e5c2ece-a4a1-464b-adba-cb217aa78780 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7662aaea 2022-04-26T06:25:37.836Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e5c2ece-a4a1-464b-adba-cb217aa78780 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:25:37.836Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e5c2ece-a4a1-464b-adba-cb217aa78780 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:25:38.045Z [qtp2085002312-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "4a83a391-0841-425b-b01b-bae244256007" INVARIANT_UUID = "434d09ad-ba73-49db-93e0-4fd09ccbf832" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=344c36fe-27ff-4fee-8a8a-f706ef70d31d TargetServiceName=null EntryTimestamp=2022-04-26T06:25:34.639Z InvocationID=9b2326fd-55bb-4502-b158-3d7e9b0ffd1b ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:38.046Z [qtp2085002312-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "4a83a391-0841-425b-b01b-bae244256007" INVARIANT_UUID = "434d09ad-ba73-49db-93e0-4fd09ccbf832" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=344c36fe-27ff-4fee-8a8a-f706ef70d31d TargetServiceName=null EntryTimestamp=2022-04-26T06:25:34.639Z InvocationID=9b2326fd-55bb-4502-b158-3d7e9b0ffd1b ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:38.146Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "4a83a391-0841-425b-b01b-bae244256007" INVARIANT_UUID = "434d09ad-ba73-49db-93e0-4fd09ccbf832" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.Compute" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=344c36fe-27ff-4fee-8a8a-f706ef70d31d TargetServiceName=null EntryTimestamp=2022-04-26T06:25:34.639Z InvocationID=9b2326fd-55bb-4502-b158-3d7e9b0ffd1b ErrorCode=500 lock resource with id a29f601a-55f6-4ac4-8136-5327cde1f81a 2022-04-26T06:25:38.305Z [qtp2085002312-14] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "4a83a391-0841-425b-b01b-bae244256007" INVARIANT_UUID = "434d09ad-ba73-49db-93e0-4fd09ccbf832" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.Compute" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=344c36fe-27ff-4fee-8a8a-f706ef70d31d TargetServiceName=null EntryTimestamp=2022-04-26T06:25:34.639Z InvocationID=9b2326fd-55bb-4502-b158-3d7e9b0ffd1b ErrorCode=500 owner of resource a29f601a-55f6-4ac4-8136-5327cde1f81a is jh0003 2022-04-26T06:25:38.858Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=6e5c2ece-a4a1-464b-adba-cb217aa78780 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:25:38.859Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=6e5c2ece-a4a1-464b-adba-cb217aa78780 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:25:40.052Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96ece5af-59fe-461c-ae42-37ed33e67d0d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:25:40.052Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96ece5af-59fe-461c-ae42-37ed33e67d0d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4cfd410f 2022-04-26T06:25:40.052Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96ece5af-59fe-461c-ae42-37ed33e67d0d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:25:40.053Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96ece5af-59fe-461c-ae42-37ed33e67d0d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:25:40.870Z [qtp2085002312-14] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "344c36fe-27ff-4fee-8a8a-f706ef70d31d" INVARIANT_UUID = "434d09ad-ba73-49db-93e0-4fd09ccbf832" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=344c36fe-27ff-4fee-8a8a-f706ef70d31d TargetServiceName=null EntryTimestamp=2022-04-26T06:25:34.639Z InvocationID=9b2326fd-55bb-4502-b158-3d7e9b0ffd1b ErrorCode=500 owner of resource a29f601a-55f6-4ac4-8136-5327cde1f81a is jh0003 2022-04-26T06:25:40.870Z [qtp2085002312-14] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "344c36fe-27ff-4fee-8a8a-f706ef70d31d" INVARIANT_UUID = "434d09ad-ba73-49db-93e0-4fd09ccbf832" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=344c36fe-27ff-4fee-8a8a-f706ef70d31d TargetServiceName=null EntryTimestamp=2022-04-26T06:25:34.639Z InvocationID=9b2326fd-55bb-4502-b158-3d7e9b0ffd1b ErrorCode=500 validate before certification change. resource name=VDU Compute, oldState=NOT_CERTIFIED_CHECKIN, owner userId=jh0003 2022-04-26T06:25:40.871Z [qtp2085002312-14] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "344c36fe-27ff-4fee-8a8a-f706ef70d31d" INVARIANT_UUID = "434d09ad-ba73-49db-93e0-4fd09ccbf832" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=344c36fe-27ff-4fee-8a8a-f706ef70d31d TargetServiceName=null EntryTimestamp=2022-04-26T06:25:34.639Z InvocationID=9b2326fd-55bb-4502-b158-3d7e9b0ffd1b ErrorCode=500 start performing certification change for resource a29f601a-55f6-4ac4-8136-5327cde1f81a 2022-04-26T06:25:41.048Z [qtp2085002312-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "344c36fe-27ff-4fee-8a8a-f706ef70d31d" INVARIANT_UUID = "434d09ad-ba73-49db-93e0-4fd09ccbf832" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "c439ea4c-eb18-4f10-98c1-37336155dd48.assettoscatemplate" CURR_ARTIFACT_UUID = "c439ea4c-eb18-4f10-98c1-37336155dd48.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCompute-template.yml,0,c439ea4c-eb18-4f10-98c1-37336155dd48.assettoscatemplate,1,62ce910d-5b28-4f30-ba35-9c4159ced199" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4b426202-4c61-42e0-811b-cacea53b8c1a TargetServiceName=null EntryTimestamp=2022-04-26T06:25:34.639Z InvocationID=9b2326fd-55bb-4502-b158-3d7e9b0ffd1b ErrorCode=500 Starting to create Csar for component VDU Compute 2022-04-26T06:25:41.066Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=96ece5af-59fe-461c-ae42-37ed33e67d0d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:25:41.066Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=96ece5af-59fe-461c-ae42-37ed33e67d0d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:25:41.208Z [qtp2085002312-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "344c36fe-27ff-4fee-8a8a-f706ef70d31d" INVARIANT_UUID = "434d09ad-ba73-49db-93e0-4fd09ccbf832" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "c439ea4c-eb18-4f10-98c1-37336155dd48.assettoscatemplate" CURR_ARTIFACT_UUID = "c439ea4c-eb18-4f10-98c1-37336155dd48.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCompute-template.yml,0,c439ea4c-eb18-4f10-98c1-37336155dd48.assettoscatemplate,1,62ce910d-5b28-4f30-ba35-9c4159ced199" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4b426202-4c61-42e0-811b-cacea53b8c1a TargetServiceName=null EntryTimestamp=2022-04-26T06:25:34.639Z InvocationID=9b2326fd-55bb-4502-b158-3d7e9b0ffd1b ErrorCode=500 Ended create Csar for component VDU Compute 2022-04-26T06:25:41.851Z [qtp2085002312-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "344c36fe-27ff-4fee-8a8a-f706ef70d31d" INVARIANT_UUID = "434d09ad-ba73-49db-93e0-4fd09ccbf832" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2a2053b5-bff5-4433-aee6-4e0da8b618c4 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:34.639Z InvocationID=9b2326fd-55bb-4502-b158-3d7e9b0ffd1b ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:41.852Z [qtp2085002312-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "344c36fe-27ff-4fee-8a8a-f706ef70d31d" INVARIANT_UUID = "434d09ad-ba73-49db-93e0-4fd09ccbf832" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2a2053b5-bff5-4433-aee6-4e0da8b618c4 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:34.639Z InvocationID=9b2326fd-55bb-4502-b158-3d7e9b0ffd1b ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:41.852Z [qtp2085002312-14] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "344c36fe-27ff-4fee-8a8a-f706ef70d31d" INVARIANT_UUID = "434d09ad-ba73-49db-93e0-4fd09ccbf832" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2a2053b5-bff5-4433-aee6-4e0da8b618c4 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:34.639Z InvocationID=9b2326fd-55bb-4502-b158-3d7e9b0ffd1b ErrorCode=500 unlock component a29f601a-55f6-4ac4-8136-5327cde1f81a 2022-04-26T06:25:41.852Z [qtp2085002312-14] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "344c36fe-27ff-4fee-8a8a-f706ef70d31d" INVARIANT_UUID = "434d09ad-ba73-49db-93e0-4fd09ccbf832" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2a2053b5-bff5-4433-aee6-4e0da8b618c4 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:34.639Z InvocationID=9b2326fd-55bb-4502-b158-3d7e9b0ffd1b ErrorCode=500 During change state, another component a29f601a-55f6-4ac4-8136-5327cde1f81a has been created/updated 2022-04-26T06:25:41.861Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "344c36fe-27ff-4fee-8a8a-f706ef70d31d" INVARIANT_UUID = "434d09ad-ba73-49db-93e0-4fd09ccbf832" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=7221 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2a2053b5-bff5-4433-aee6-4e0da8b618c4 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:34.639Z InvocationID=9b2326fd-55bb-4502-b158-3d7e9b0ffd1b ErrorCode=500 LogTimestamp=2022-04-26T06:25:41.860926Z ResponseDesc=Created Exiting. 2022-04-26T06:25:41.861Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:25:41.873Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=27580d2f-3345-4b97-9352-86c70d186dc3 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:41.873097Z InvocationID=3c0d9b4f-5214-4fbc-a8bd-ace3f5ea37d3 LogTimestamp=2022-04-26T06:25:41.873198Z Entering 2022-04-26T06:25:41.873Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=27580d2f-3345-4b97-9352-86c70d186dc3 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:41.873097Z InvocationID=3c0d9b4f-5214-4fbc-a8bd-ace3f5ea37d3 LogTimestamp=2022-04-26T06:25:41.873198Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:25:42.011Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:41.874Z InvocationID=68aacc2b-6ddc-4ab9-bf0c-31ce7bb652c8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id cf3efd4e-334d-409f-8a1d-f7bbbf2479aa 2022-04-26T06:25:42.026Z [qtp2085002312-15] INFO o.o.s.b.c.l.LifecycleBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:41.874Z InvocationID=68aacc2b-6ddc-4ab9-bf0c-31ce7bb652c8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb owner of resource cf3efd4e-334d-409f-8a1d-f7bbbf2479aa is jh0003 2022-04-26T06:25:42.074Z [qtp2085002312-15] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "06b18a6d-87d6-45a8-868e-078256ad4257" INVARIANT_UUID = "125d1e27-af6c-4dd1-af02-c72d377c47c5" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:41.874Z InvocationID=68aacc2b-6ddc-4ab9-bf0c-31ce7bb652c8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:42.075Z [qtp2085002312-15] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "06b18a6d-87d6-45a8-868e-078256ad4257" INVARIANT_UUID = "125d1e27-af6c-4dd1-af02-c72d377c47c5" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:25:41.874Z InvocationID=68aacc2b-6ddc-4ab9-bf0c-31ce7bb652c8 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:42.221Z [qtp2085002312-15] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "06b18a6d-87d6-45a8-868e-078256ad4257" INVARIANT_UUID = "125d1e27-af6c-4dd1-af02-c72d377c47c5" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d07a21d0-4bbf-4c17-a002-ac26feedc90c TargetServiceName=null EntryTimestamp=2022-04-26T06:25:41.874Z InvocationID=68aacc2b-6ddc-4ab9-bf0c-31ce7bb652c8 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:42.222Z [qtp2085002312-15] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "06b18a6d-87d6-45a8-868e-078256ad4257" INVARIANT_UUID = "125d1e27-af6c-4dd1-af02-c72d377c47c5" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d07a21d0-4bbf-4c17-a002-ac26feedc90c TargetServiceName=null EntryTimestamp=2022-04-26T06:25:41.874Z InvocationID=68aacc2b-6ddc-4ab9-bf0c-31ce7bb652c8 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:42.250Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "06b18a6d-87d6-45a8-868e-078256ad4257" INVARIANT_UUID = "125d1e27-af6c-4dd1-af02-c72d377c47c5" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualFileStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d07a21d0-4bbf-4c17-a002-ac26feedc90c TargetServiceName=null EntryTimestamp=2022-04-26T06:25:41.874Z InvocationID=68aacc2b-6ddc-4ab9-bf0c-31ce7bb652c8 ErrorCode=500 lock resource with id 90c953a7-635e-4526-b76d-6abae6ba342c 2022-04-26T06:25:42.253Z [qtp2085002312-15] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "06b18a6d-87d6-45a8-868e-078256ad4257" INVARIANT_UUID = "125d1e27-af6c-4dd1-af02-c72d377c47c5" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualFileStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d07a21d0-4bbf-4c17-a002-ac26feedc90c TargetServiceName=null EntryTimestamp=2022-04-26T06:25:41.874Z InvocationID=68aacc2b-6ddc-4ab9-bf0c-31ce7bb652c8 ErrorCode=500 owner of resource 90c953a7-635e-4526-b76d-6abae6ba342c is jh0003 2022-04-26T06:25:42.798Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=272d45bc-f01e-4316-ae5d-ee720ca26743 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:25:42.798Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=272d45bc-f01e-4316-ae5d-ee720ca26743 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@44fc3620 2022-04-26T06:25:42.798Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=272d45bc-f01e-4316-ae5d-ee720ca26743 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:25:42.798Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=272d45bc-f01e-4316-ae5d-ee720ca26743 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:25:42.897Z [qtp2085002312-15] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d07a21d0-4bbf-4c17-a002-ac26feedc90c" INVARIANT_UUID = "125d1e27-af6c-4dd1-af02-c72d377c47c5" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d07a21d0-4bbf-4c17-a002-ac26feedc90c TargetServiceName=null EntryTimestamp=2022-04-26T06:25:41.874Z InvocationID=68aacc2b-6ddc-4ab9-bf0c-31ce7bb652c8 ErrorCode=500 owner of resource 90c953a7-635e-4526-b76d-6abae6ba342c is jh0003 2022-04-26T06:25:42.897Z [qtp2085002312-15] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d07a21d0-4bbf-4c17-a002-ac26feedc90c" INVARIANT_UUID = "125d1e27-af6c-4dd1-af02-c72d377c47c5" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d07a21d0-4bbf-4c17-a002-ac26feedc90c TargetServiceName=null EntryTimestamp=2022-04-26T06:25:41.874Z InvocationID=68aacc2b-6ddc-4ab9-bf0c-31ce7bb652c8 ErrorCode=500 validate before certification change. resource name=VDU VirtualFileStorage, oldState=NOT_CERTIFIED_CHECKIN, owner userId=jh0003 2022-04-26T06:25:42.897Z [qtp2085002312-15] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d07a21d0-4bbf-4c17-a002-ac26feedc90c" INVARIANT_UUID = "125d1e27-af6c-4dd1-af02-c72d377c47c5" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d07a21d0-4bbf-4c17-a002-ac26feedc90c TargetServiceName=null EntryTimestamp=2022-04-26T06:25:41.874Z InvocationID=68aacc2b-6ddc-4ab9-bf0c-31ce7bb652c8 ErrorCode=500 start performing certification change for resource 90c953a7-635e-4526-b76d-6abae6ba342c 2022-04-26T06:25:43.034Z [qtp2085002312-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d07a21d0-4bbf-4c17-a002-ac26feedc90c" INVARIANT_UUID = "125d1e27-af6c-4dd1-af02-c72d377c47c5" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "cf3efd4e-334d-409f-8a1d-f7bbbf2479aa.assettoscatemplate" CURR_ARTIFACT_UUID = "cf3efd4e-334d-409f-8a1d-f7bbbf2479aa.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualfilestorage-template.yml,0,cf3efd4e-334d-409f-8a1d-f7bbbf2479aa.assettoscatemplate,1,e4a20ebb-0e3e-4af4-85f7-c977b034408c" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1a11b996-b158-43ef-bb96-815785a419e3 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:41.874Z InvocationID=68aacc2b-6ddc-4ab9-bf0c-31ce7bb652c8 ErrorCode=500 Starting to create Csar for component VDU VirtualFileStorage 2022-04-26T06:25:43.197Z [qtp2085002312-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d07a21d0-4bbf-4c17-a002-ac26feedc90c" INVARIANT_UUID = "125d1e27-af6c-4dd1-af02-c72d377c47c5" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "cf3efd4e-334d-409f-8a1d-f7bbbf2479aa.assettoscatemplate" CURR_ARTIFACT_UUID = "cf3efd4e-334d-409f-8a1d-f7bbbf2479aa.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualfilestorage-template.yml,0,cf3efd4e-334d-409f-8a1d-f7bbbf2479aa.assettoscatemplate,1,e4a20ebb-0e3e-4af4-85f7-c977b034408c" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1a11b996-b158-43ef-bb96-815785a419e3 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:41.874Z InvocationID=68aacc2b-6ddc-4ab9-bf0c-31ce7bb652c8 ErrorCode=500 Ended create Csar for component VDU VirtualFileStorage 2022-04-26T06:25:43.817Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=272d45bc-f01e-4316-ae5d-ee720ca26743 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:25:43.817Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=272d45bc-f01e-4316-ae5d-ee720ca26743 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:25:43.846Z [qtp2085002312-15] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d07a21d0-4bbf-4c17-a002-ac26feedc90c" INVARIANT_UUID = "125d1e27-af6c-4dd1-af02-c72d377c47c5" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f6b94211-95b5-4fea-aa05-728518cba864 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:41.874Z InvocationID=68aacc2b-6ddc-4ab9-bf0c-31ce7bb652c8 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:43.847Z [qtp2085002312-15] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d07a21d0-4bbf-4c17-a002-ac26feedc90c" INVARIANT_UUID = "125d1e27-af6c-4dd1-af02-c72d377c47c5" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f6b94211-95b5-4fea-aa05-728518cba864 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:41.874Z InvocationID=68aacc2b-6ddc-4ab9-bf0c-31ce7bb652c8 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:43.847Z [qtp2085002312-15] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d07a21d0-4bbf-4c17-a002-ac26feedc90c" INVARIANT_UUID = "125d1e27-af6c-4dd1-af02-c72d377c47c5" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f6b94211-95b5-4fea-aa05-728518cba864 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:41.874Z InvocationID=68aacc2b-6ddc-4ab9-bf0c-31ce7bb652c8 ErrorCode=500 unlock component 90c953a7-635e-4526-b76d-6abae6ba342c 2022-04-26T06:25:43.847Z [qtp2085002312-15] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d07a21d0-4bbf-4c17-a002-ac26feedc90c" INVARIANT_UUID = "125d1e27-af6c-4dd1-af02-c72d377c47c5" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f6b94211-95b5-4fea-aa05-728518cba864 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:41.874Z InvocationID=68aacc2b-6ddc-4ab9-bf0c-31ce7bb652c8 ErrorCode=500 During change state, another component 90c953a7-635e-4526-b76d-6abae6ba342c has been created/updated 2022-04-26T06:25:43.854Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d07a21d0-4bbf-4c17-a002-ac26feedc90c" INVARIANT_UUID = "125d1e27-af6c-4dd1-af02-c72d377c47c5" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1980 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f6b94211-95b5-4fea-aa05-728518cba864 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:41.874Z InvocationID=68aacc2b-6ddc-4ab9-bf0c-31ce7bb652c8 ErrorCode=500 LogTimestamp=2022-04-26T06:25:43.854323Z ResponseDesc=Created Exiting. 2022-04-26T06:25:43.855Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:25:43.864Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=40d2d47f-c197-4910-b752-9af9e7499ca6 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:43.863810Z InvocationID=94977bf8-f60b-4dfa-9031-4e148534014b LogTimestamp=2022-04-26T06:25:43.863878Z Entering 2022-04-26T06:25:43.864Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=40d2d47f-c197-4910-b752-9af9e7499ca6 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:43.863810Z InvocationID=94977bf8-f60b-4dfa-9031-4e148534014b LogTimestamp=2022-04-26T06:25:43.863878Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:25:43.993Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:43.864Z InvocationID=07d7a6c4-5cdf-4b91-830f-cf1d9c3ea818 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 5e77c04e-396c-48b0-93af-09b775df8cb0 2022-04-26T06:25:44.005Z [qtp2085002312-20] INFO o.o.s.b.c.l.LifecycleBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:43.864Z InvocationID=07d7a6c4-5cdf-4b91-830f-cf1d9c3ea818 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb owner of resource 5e77c04e-396c-48b0-93af-09b775df8cb0 is jh0003 2022-04-26T06:25:44.040Z [qtp2085002312-20] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "2e016464-fe66-4dea-a43e-6e741c944b4d" INVARIANT_UUID = "20e54cf4-2d55-4e9b-9d10-440bb37b149c" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:43.864Z InvocationID=07d7a6c4-5cdf-4b91-830f-cf1d9c3ea818 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:44.041Z [qtp2085002312-20] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "2e016464-fe66-4dea-a43e-6e741c944b4d" INVARIANT_UUID = "20e54cf4-2d55-4e9b-9d10-440bb37b149c" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:25:43.864Z InvocationID=07d7a6c4-5cdf-4b91-830f-cf1d9c3ea818 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:44.167Z [qtp2085002312-20] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "2e016464-fe66-4dea-a43e-6e741c944b4d" INVARIANT_UUID = "20e54cf4-2d55-4e9b-9d10-440bb37b149c" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2614b9df-31ac-4f93-9ded-6d4aa4bb88ac TargetServiceName=null EntryTimestamp=2022-04-26T06:25:43.864Z InvocationID=07d7a6c4-5cdf-4b91-830f-cf1d9c3ea818 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:44.167Z [qtp2085002312-20] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "2e016464-fe66-4dea-a43e-6e741c944b4d" INVARIANT_UUID = "20e54cf4-2d55-4e9b-9d10-440bb37b149c" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2614b9df-31ac-4f93-9ded-6d4aa4bb88ac TargetServiceName=null EntryTimestamp=2022-04-26T06:25:43.864Z InvocationID=07d7a6c4-5cdf-4b91-830f-cf1d9c3ea818 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:44.177Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "2e016464-fe66-4dea-a43e-6e741c944b4d" INVARIANT_UUID = "20e54cf4-2d55-4e9b-9d10-440bb37b149c" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VnfExtCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2614b9df-31ac-4f93-9ded-6d4aa4bb88ac TargetServiceName=null EntryTimestamp=2022-04-26T06:25:43.864Z InvocationID=07d7a6c4-5cdf-4b91-830f-cf1d9c3ea818 ErrorCode=500 lock resource with id 6bb6d2be-6996-4843-bdc8-c8ed3bda802a 2022-04-26T06:25:44.178Z [qtp2085002312-20] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "2e016464-fe66-4dea-a43e-6e741c944b4d" INVARIANT_UUID = "20e54cf4-2d55-4e9b-9d10-440bb37b149c" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VnfExtCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2614b9df-31ac-4f93-9ded-6d4aa4bb88ac TargetServiceName=null EntryTimestamp=2022-04-26T06:25:43.864Z InvocationID=07d7a6c4-5cdf-4b91-830f-cf1d9c3ea818 ErrorCode=500 owner of resource 6bb6d2be-6996-4843-bdc8-c8ed3bda802a is jh0003 2022-04-26T06:25:44.251Z [cluster1-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:25:44.252Z [cluster1-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 16000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:25:44.858Z [qtp2085002312-20] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "2614b9df-31ac-4f93-9ded-6d4aa4bb88ac" INVARIANT_UUID = "20e54cf4-2d55-4e9b-9d10-440bb37b149c" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2614b9df-31ac-4f93-9ded-6d4aa4bb88ac TargetServiceName=null EntryTimestamp=2022-04-26T06:25:43.864Z InvocationID=07d7a6c4-5cdf-4b91-830f-cf1d9c3ea818 ErrorCode=500 owner of resource 6bb6d2be-6996-4843-bdc8-c8ed3bda802a is jh0003 2022-04-26T06:25:44.858Z [qtp2085002312-20] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "2614b9df-31ac-4f93-9ded-6d4aa4bb88ac" INVARIANT_UUID = "20e54cf4-2d55-4e9b-9d10-440bb37b149c" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2614b9df-31ac-4f93-9ded-6d4aa4bb88ac TargetServiceName=null EntryTimestamp=2022-04-26T06:25:43.864Z InvocationID=07d7a6c4-5cdf-4b91-830f-cf1d9c3ea818 ErrorCode=500 validate before certification change. resource name=VNF External CP, oldState=NOT_CERTIFIED_CHECKIN, owner userId=jh0003 2022-04-26T06:25:44.858Z [qtp2085002312-20] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "2614b9df-31ac-4f93-9ded-6d4aa4bb88ac" INVARIANT_UUID = "20e54cf4-2d55-4e9b-9d10-440bb37b149c" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2614b9df-31ac-4f93-9ded-6d4aa4bb88ac TargetServiceName=null EntryTimestamp=2022-04-26T06:25:43.864Z InvocationID=07d7a6c4-5cdf-4b91-830f-cf1d9c3ea818 ErrorCode=500 start performing certification change for resource 6bb6d2be-6996-4843-bdc8-c8ed3bda802a 2022-04-26T06:25:44.950Z [JanusGraph Cluster-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:25:44.951Z [JanusGraph Cluster-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 8000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:25:44.990Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "2614b9df-31ac-4f93-9ded-6d4aa4bb88ac" INVARIANT_UUID = "20e54cf4-2d55-4e9b-9d10-440bb37b149c" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "5e77c04e-396c-48b0-93af-09b775df8cb0.assettoscatemplate" CURR_ARTIFACT_UUID = "5e77c04e-396c-48b0-93af-09b775df8cb0.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VnfExternalCp-template.yml,0,5e77c04e-396c-48b0-93af-09b775df8cb0.assettoscatemplate,1,3892d571-cc1e-4e3d-b3b8-78d2f92f348a" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=371c84b7-05a5-4a42-bf6d-baa8bbdae4ee TargetServiceName=null EntryTimestamp=2022-04-26T06:25:43.864Z InvocationID=07d7a6c4-5cdf-4b91-830f-cf1d9c3ea818 ErrorCode=500 Starting to create Csar for component VNF External CP 2022-04-26T06:25:45.204Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "2614b9df-31ac-4f93-9ded-6d4aa4bb88ac" INVARIANT_UUID = "20e54cf4-2d55-4e9b-9d10-440bb37b149c" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "5e77c04e-396c-48b0-93af-09b775df8cb0.assettoscatemplate" CURR_ARTIFACT_UUID = "5e77c04e-396c-48b0-93af-09b775df8cb0.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VnfExternalCp-template.yml,0,5e77c04e-396c-48b0-93af-09b775df8cb0.assettoscatemplate,1,3892d571-cc1e-4e3d-b3b8-78d2f92f348a" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=371c84b7-05a5-4a42-bf6d-baa8bbdae4ee TargetServiceName=null EntryTimestamp=2022-04-26T06:25:43.864Z InvocationID=07d7a6c4-5cdf-4b91-830f-cf1d9c3ea818 ErrorCode=500 Ended create Csar for component VNF External CP 2022-04-26T06:25:45.792Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c24255d-eb61-4cdb-85e9-4ce05f756548 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:25:45.792Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c24255d-eb61-4cdb-85e9-4ce05f756548 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e641247 2022-04-26T06:25:45.792Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c24255d-eb61-4cdb-85e9-4ce05f756548 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:25:45.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c24255d-eb61-4cdb-85e9-4ce05f756548 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:25:45.866Z [qtp2085002312-20] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "2614b9df-31ac-4f93-9ded-6d4aa4bb88ac" INVARIANT_UUID = "20e54cf4-2d55-4e9b-9d10-440bb37b149c" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=da5fb226-0fb6-43a0-aad1-d60aa188ce38 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:43.864Z InvocationID=07d7a6c4-5cdf-4b91-830f-cf1d9c3ea818 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:45.867Z [qtp2085002312-20] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "2614b9df-31ac-4f93-9ded-6d4aa4bb88ac" INVARIANT_UUID = "20e54cf4-2d55-4e9b-9d10-440bb37b149c" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=da5fb226-0fb6-43a0-aad1-d60aa188ce38 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:43.864Z InvocationID=07d7a6c4-5cdf-4b91-830f-cf1d9c3ea818 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:45.867Z [qtp2085002312-20] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "2614b9df-31ac-4f93-9ded-6d4aa4bb88ac" INVARIANT_UUID = "20e54cf4-2d55-4e9b-9d10-440bb37b149c" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=da5fb226-0fb6-43a0-aad1-d60aa188ce38 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:43.864Z InvocationID=07d7a6c4-5cdf-4b91-830f-cf1d9c3ea818 ErrorCode=500 unlock component 6bb6d2be-6996-4843-bdc8-c8ed3bda802a 2022-04-26T06:25:45.867Z [qtp2085002312-20] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "2614b9df-31ac-4f93-9ded-6d4aa4bb88ac" INVARIANT_UUID = "20e54cf4-2d55-4e9b-9d10-440bb37b149c" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=da5fb226-0fb6-43a0-aad1-d60aa188ce38 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:43.864Z InvocationID=07d7a6c4-5cdf-4b91-830f-cf1d9c3ea818 ErrorCode=500 During change state, another component 6bb6d2be-6996-4843-bdc8-c8ed3bda802a has been created/updated 2022-04-26T06:25:45.878Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "2614b9df-31ac-4f93-9ded-6d4aa4bb88ac" INVARIANT_UUID = "20e54cf4-2d55-4e9b-9d10-440bb37b149c" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=2013 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=da5fb226-0fb6-43a0-aad1-d60aa188ce38 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:43.864Z InvocationID=07d7a6c4-5cdf-4b91-830f-cf1d9c3ea818 ErrorCode=500 LogTimestamp=2022-04-26T06:25:45.877452Z ResponseDesc=Created Exiting. 2022-04-26T06:25:45.878Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:25:45.889Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=35428c8b-0c37-4f07-9b12-6ac576e6cdc1 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:45.888667Z InvocationID=36177452-6aa2-4c40-a9a1-138f9ecae196 LogTimestamp=2022-04-26T06:25:45.888728Z Entering 2022-04-26T06:25:45.889Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=35428c8b-0c37-4f07-9b12-6ac576e6cdc1 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:45.888667Z InvocationID=36177452-6aa2-4c40-a9a1-138f9ecae196 LogTimestamp=2022-04-26T06:25:45.888728Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:25:45.982Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:45.889Z InvocationID=7e910228-2c79-458e-8ebc-3450558bb9ee PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 6c919be2-e998-47c6-af28-39cdf81a48d7 2022-04-26T06:25:45.993Z [qtp2085002312-15] INFO o.o.s.b.c.l.LifecycleBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:45.889Z InvocationID=7e910228-2c79-458e-8ebc-3450558bb9ee PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb owner of resource 6c919be2-e998-47c6-af28-39cdf81a48d7 is jh0003 2022-04-26T06:25:46.032Z [qtp2085002312-15] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "dda2062d-0a04-4330-97bd-6d8521af2e18" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:45.889Z InvocationID=7e910228-2c79-458e-8ebc-3450558bb9ee PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:46.033Z [qtp2085002312-15] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "dda2062d-0a04-4330-97bd-6d8521af2e18" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:25:45.889Z InvocationID=7e910228-2c79-458e-8ebc-3450558bb9ee ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:46.142Z [qtp2085002312-15] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "dda2062d-0a04-4330-97bd-6d8521af2e18" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=35061a8e-98c7-4793-aec0-da835215fb4b TargetServiceName=null EntryTimestamp=2022-04-26T06:25:45.889Z InvocationID=7e910228-2c79-458e-8ebc-3450558bb9ee ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:46.143Z [qtp2085002312-15] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "dda2062d-0a04-4330-97bd-6d8521af2e18" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=35061a8e-98c7-4793-aec0-da835215fb4b TargetServiceName=null EntryTimestamp=2022-04-26T06:25:45.889Z InvocationID=7e910228-2c79-458e-8ebc-3450558bb9ee ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:46.155Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "dda2062d-0a04-4330-97bd-6d8521af2e18" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VduCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=35061a8e-98c7-4793-aec0-da835215fb4b TargetServiceName=null EntryTimestamp=2022-04-26T06:25:45.889Z InvocationID=7e910228-2c79-458e-8ebc-3450558bb9ee ErrorCode=500 lock resource with id df70bdb7-2401-42e1-973a-22c27680e7a3 2022-04-26T06:25:46.157Z [qtp2085002312-15] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "dda2062d-0a04-4330-97bd-6d8521af2e18" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VduCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=35061a8e-98c7-4793-aec0-da835215fb4b TargetServiceName=null EntryTimestamp=2022-04-26T06:25:45.889Z InvocationID=7e910228-2c79-458e-8ebc-3450558bb9ee ErrorCode=500 owner of resource df70bdb7-2401-42e1-973a-22c27680e7a3 is jh0003 2022-04-26T06:25:46.794Z [qtp2085002312-15] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "35061a8e-98c7-4793-aec0-da835215fb4b" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=35061a8e-98c7-4793-aec0-da835215fb4b TargetServiceName=null EntryTimestamp=2022-04-26T06:25:45.889Z InvocationID=7e910228-2c79-458e-8ebc-3450558bb9ee ErrorCode=500 owner of resource df70bdb7-2401-42e1-973a-22c27680e7a3 is jh0003 2022-04-26T06:25:46.795Z [qtp2085002312-15] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "35061a8e-98c7-4793-aec0-da835215fb4b" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=35061a8e-98c7-4793-aec0-da835215fb4b TargetServiceName=null EntryTimestamp=2022-04-26T06:25:45.889Z InvocationID=7e910228-2c79-458e-8ebc-3450558bb9ee ErrorCode=500 validate before certification change. resource name=VDU Cp, oldState=NOT_CERTIFIED_CHECKIN, owner userId=jh0003 2022-04-26T06:25:46.795Z [qtp2085002312-15] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "35061a8e-98c7-4793-aec0-da835215fb4b" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=35061a8e-98c7-4793-aec0-da835215fb4b TargetServiceName=null EntryTimestamp=2022-04-26T06:25:45.889Z InvocationID=7e910228-2c79-458e-8ebc-3450558bb9ee ErrorCode=500 start performing certification change for resource df70bdb7-2401-42e1-973a-22c27680e7a3 2022-04-26T06:25:46.831Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=7c24255d-eb61-4cdb-85e9-4ce05f756548 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:25:46.832Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=7c24255d-eb61-4cdb-85e9-4ce05f756548 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:25:46.925Z [qtp2085002312-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "35061a8e-98c7-4793-aec0-da835215fb4b" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "6c919be2-e998-47c6-af28-39cdf81a48d7.assettoscatemplate" CURR_ARTIFACT_UUID = "6c919be2-e998-47c6-af28-39cdf81a48d7.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCp-template.yml,0,6c919be2-e998-47c6-af28-39cdf81a48d7.assettoscatemplate,1,14400f60-98f4-44b7-9b44-ed0433e547f7" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e576364e-d726-463d-af32-c339fb656b9c TargetServiceName=null EntryTimestamp=2022-04-26T06:25:45.889Z InvocationID=7e910228-2c79-458e-8ebc-3450558bb9ee ErrorCode=500 Starting to create Csar for component VDU Cp 2022-04-26T06:25:47.083Z [qtp2085002312-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "35061a8e-98c7-4793-aec0-da835215fb4b" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "6c919be2-e998-47c6-af28-39cdf81a48d7.assettoscatemplate" CURR_ARTIFACT_UUID = "6c919be2-e998-47c6-af28-39cdf81a48d7.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCp-template.yml,0,6c919be2-e998-47c6-af28-39cdf81a48d7.assettoscatemplate,1,14400f60-98f4-44b7-9b44-ed0433e547f7" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e576364e-d726-463d-af32-c339fb656b9c TargetServiceName=null EntryTimestamp=2022-04-26T06:25:45.889Z InvocationID=7e910228-2c79-458e-8ebc-3450558bb9ee ErrorCode=500 Ended create Csar for component VDU Cp 2022-04-26T06:25:47.699Z [qtp2085002312-15] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "35061a8e-98c7-4793-aec0-da835215fb4b" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c8939e30-ebac-428c-a7b7-553206d45b34 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:45.889Z InvocationID=7e910228-2c79-458e-8ebc-3450558bb9ee ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:47.699Z [qtp2085002312-15] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "35061a8e-98c7-4793-aec0-da835215fb4b" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c8939e30-ebac-428c-a7b7-553206d45b34 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:45.889Z InvocationID=7e910228-2c79-458e-8ebc-3450558bb9ee ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:47.700Z [qtp2085002312-15] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "35061a8e-98c7-4793-aec0-da835215fb4b" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c8939e30-ebac-428c-a7b7-553206d45b34 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:45.889Z InvocationID=7e910228-2c79-458e-8ebc-3450558bb9ee ErrorCode=500 unlock component df70bdb7-2401-42e1-973a-22c27680e7a3 2022-04-26T06:25:47.700Z [qtp2085002312-15] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "35061a8e-98c7-4793-aec0-da835215fb4b" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c8939e30-ebac-428c-a7b7-553206d45b34 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:45.889Z InvocationID=7e910228-2c79-458e-8ebc-3450558bb9ee ErrorCode=500 During change state, another component df70bdb7-2401-42e1-973a-22c27680e7a3 has been created/updated 2022-04-26T06:25:47.710Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "35061a8e-98c7-4793-aec0-da835215fb4b" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1821 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c8939e30-ebac-428c-a7b7-553206d45b34 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:45.889Z InvocationID=7e910228-2c79-458e-8ebc-3450558bb9ee ErrorCode=500 LogTimestamp=2022-04-26T06:25:47.710137Z ResponseDesc=Created Exiting. 2022-04-26T06:25:47.711Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:25:47.741Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=884511a6-234e-4f71-ae19-e0ef0f2136b6 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:47.741069Z InvocationID=10bd7d3f-62f6-41c0-8436-5a46b9f6ff86 LogTimestamp=2022-04-26T06:25:47.741137Z Entering 2022-04-26T06:25:47.741Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=884511a6-234e-4f71-ae19-e0ef0f2136b6 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:47.741069Z InvocationID=10bd7d3f-62f6-41c0-8436-5a46b9f6ff86 LogTimestamp=2022-04-26T06:25:47.741137Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:25:47.833Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:47.742Z InvocationID=10f44e13-ae7e-40aa-a34e-543d279f9c47 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:47.834Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:47.742Z InvocationID=10f44e13-ae7e-40aa-a34e-543d279f9c47 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:47.834Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:47.742Z InvocationID=10f44e13-ae7e-40aa-a34e-543d279f9c47 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:47.834Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:47.742Z InvocationID=10f44e13-ae7e-40aa-a34e-543d279f9c47 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:25:47.842Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:47.742Z InvocationID=10f44e13-ae7e-40aa-a34e-543d279f9c47 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name VipCp 2022-04-26T06:25:47.887Z [qtp2085002312-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VIP Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "25ee6084-679c-44c4-9711-7450e08cce50" INVARIANT_UUID = "e6b5143f-076f-4e60-9465-aad0353b44b7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VipCp" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=25ee6084-679c-44c4-9711-7450e08cce50 EntryTimestamp=2022-04-26T06:25:47.742Z InvocationID=10f44e13-ae7e-40aa-a34e-543d279f9c47 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:47.887Z [qtp2085002312-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VIP Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "25ee6084-679c-44c4-9711-7450e08cce50" INVARIANT_UUID = "e6b5143f-076f-4e60-9465-aad0353b44b7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VipCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=25ee6084-679c-44c4-9711-7450e08cce50 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:47.742Z InvocationID=10f44e13-ae7e-40aa-a34e-543d279f9c47 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:47.934Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VIP Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "25ee6084-679c-44c4-9711-7450e08cce50" INVARIANT_UUID = "e6b5143f-076f-4e60-9465-aad0353b44b7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VipCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=25ee6084-679c-44c4-9711-7450e08cce50 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:47.742Z InvocationID=10f44e13-ae7e-40aa-a34e-543d279f9c47 ErrorCode=500 lock resource with id e5a93a81-04c3-476d-bac4-c5e1c38055f1 2022-04-26T06:25:48.555Z [qtp2085002312-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VIP Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "25ee6084-679c-44c4-9711-7450e08cce50" INVARIANT_UUID = "e6b5143f-076f-4e60-9465-aad0353b44b7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "e5a93a81-04c3-476d-bac4-c5e1c38055f1.assettoscatemplate" CURR_ARTIFACT_UUID = "e5a93a81-04c3-476d-bac4-c5e1c38055f1.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VipCp-template.yml,0,e5a93a81-04c3-476d-bac4-c5e1c38055f1.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=88f24e32-ee9e-4b32-b739-ad7c808c18e5 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:47.742Z InvocationID=10f44e13-ae7e-40aa-a34e-543d279f9c47 ErrorCode=500 Starting to create Csar for component VIP Cp 2022-04-26T06:25:48.746Z [qtp2085002312-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VIP Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "25ee6084-679c-44c4-9711-7450e08cce50" INVARIANT_UUID = "e6b5143f-076f-4e60-9465-aad0353b44b7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "e5a93a81-04c3-476d-bac4-c5e1c38055f1.assettoscatemplate" CURR_ARTIFACT_UUID = "e5a93a81-04c3-476d-bac4-c5e1c38055f1.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VipCp-template.yml,0,e5a93a81-04c3-476d-bac4-c5e1c38055f1.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=88f24e32-ee9e-4b32-b739-ad7c808c18e5 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:47.742Z InvocationID=10f44e13-ae7e-40aa-a34e-543d279f9c47 ErrorCode=500 Ended create Csar for component VIP Cp 2022-04-26T06:25:48.746Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VIP Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "25ee6084-679c-44c4-9711-7450e08cce50" INVARIANT_UUID = "e6b5143f-076f-4e60-9465-aad0353b44b7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "e5a93a81-04c3-476d-bac4-c5e1c38055f1.assettoscatemplate" CURR_ARTIFACT_UUID = "e5a93a81-04c3-476d-bac4-c5e1c38055f1.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VipCp-template.yml,0,e5a93a81-04c3-476d-bac4-c5e1c38055f1.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=88f24e32-ee9e-4b32-b739-ad7c808c18e5 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:47.742Z InvocationID=10f44e13-ae7e-40aa-a34e-543d279f9c47 ErrorCode=500 lock resource with id e5a93a81-04c3-476d-bac4-c5e1c38055f1 2022-04-26T06:25:48.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4bebc9a3-36f0-4f5f-ad5f-caf43f5c02a4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:25:48.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4bebc9a3-36f0-4f5f-ad5f-caf43f5c02a4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2bb8f6c7 2022-04-26T06:25:48.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4bebc9a3-36f0-4f5f-ad5f-caf43f5c02a4 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:25:48.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4bebc9a3-36f0-4f5f-ad5f-caf43f5c02a4 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:25:49.347Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VIP Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "25ee6084-679c-44c4-9711-7450e08cce50" INVARIANT_UUID = "e6b5143f-076f-4e60-9465-aad0353b44b7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "e5a93a81-04c3-476d-bac4-c5e1c38055f1.assettoscacsar" CURR_ARTIFACT_UUID = "e5a93a81-04c3-476d-bac4-c5e1c38055f1.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VipCp-csar.csar,0,e5a93a81-04c3-476d-bac4-c5e1c38055f1.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1605 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e311d1c7-fe79-4dc2-a4d7-512e7f3e5da9 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:47.742Z InvocationID=10f44e13-ae7e-40aa-a34e-543d279f9c47 ErrorCode=500 LogTimestamp=2022-04-26T06:25:49.347351Z ResponseDesc=Created Exiting. 2022-04-26T06:25:49.348Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:25:49.361Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c7e03580-e081-46a6-934b-624eaebdad16 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:49.360913Z InvocationID=87441fcf-6aca-49fb-98b8-72ec7433486a LogTimestamp=2022-04-26T06:25:49.360974Z Entering 2022-04-26T06:25:49.361Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c7e03580-e081-46a6-934b-624eaebdad16 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:49.360913Z InvocationID=87441fcf-6aca-49fb-98b8-72ec7433486a LogTimestamp=2022-04-26T06:25:49.360974Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:25:49.741Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:49.361Z InvocationID=8f3c36e9-1e02-44bc-94b9-c632dec4a50e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 44465c5d-d82b-406e-9f0c-9731e383f4d6 2022-04-26T06:25:49.752Z [qtp2085002312-17] INFO o.o.s.b.c.l.LifecycleBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:49.361Z InvocationID=8f3c36e9-1e02-44bc-94b9-c632dec4a50e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb owner of resource 44465c5d-d82b-406e-9f0c-9731e383f4d6 is jh0003 2022-04-26T06:25:49.806Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=4bebc9a3-36f0-4f5f-ad5f-caf43f5c02a4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:25:49.807Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=4bebc9a3-36f0-4f5f-ad5f-caf43f5c02a4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:25:49.858Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "05dd51de-6c1f-401f-9c37-0a95ee4e11b6" INVARIANT_UUID = "b006ae6a-aad8-4be2-9315-2c995dbd801e" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:49.361Z InvocationID=8f3c36e9-1e02-44bc-94b9-c632dec4a50e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:49.859Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "05dd51de-6c1f-401f-9c37-0a95ee4e11b6" INVARIANT_UUID = "b006ae6a-aad8-4be2-9315-2c995dbd801e" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:25:49.361Z InvocationID=8f3c36e9-1e02-44bc-94b9-c632dec4a50e ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:50.340Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "05dd51de-6c1f-401f-9c37-0a95ee4e11b6" INVARIANT_UUID = "b006ae6a-aad8-4be2-9315-2c995dbd801e" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9c908063-6335-4571-b949-068e8b6f409f TargetServiceName=null EntryTimestamp=2022-04-26T06:25:49.361Z InvocationID=8f3c36e9-1e02-44bc-94b9-c632dec4a50e ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:50.342Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "05dd51de-6c1f-401f-9c37-0a95ee4e11b6" INVARIANT_UUID = "b006ae6a-aad8-4be2-9315-2c995dbd801e" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9c908063-6335-4571-b949-068e8b6f409f TargetServiceName=null EntryTimestamp=2022-04-26T06:25:49.361Z InvocationID=8f3c36e9-1e02-44bc-94b9-c632dec4a50e ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:50.351Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "05dd51de-6c1f-401f-9c37-0a95ee4e11b6" INVARIANT_UUID = "b006ae6a-aad8-4be2-9315-2c995dbd801e" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualBlockStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9c908063-6335-4571-b949-068e8b6f409f TargetServiceName=null EntryTimestamp=2022-04-26T06:25:49.361Z InvocationID=8f3c36e9-1e02-44bc-94b9-c632dec4a50e ErrorCode=500 lock resource with id 88a8a792-7764-4286-b404-e7a2a15b87e4 2022-04-26T06:25:50.353Z [qtp2085002312-17] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "05dd51de-6c1f-401f-9c37-0a95ee4e11b6" INVARIANT_UUID = "b006ae6a-aad8-4be2-9315-2c995dbd801e" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualBlockStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9c908063-6335-4571-b949-068e8b6f409f TargetServiceName=null EntryTimestamp=2022-04-26T06:25:49.361Z InvocationID=8f3c36e9-1e02-44bc-94b9-c632dec4a50e ErrorCode=500 owner of resource 88a8a792-7764-4286-b404-e7a2a15b87e4 is jh0003 2022-04-26T06:25:50.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:25:50.374Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:25:51.649Z [qtp2085002312-17] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "9c908063-6335-4571-b949-068e8b6f409f" INVARIANT_UUID = "b006ae6a-aad8-4be2-9315-2c995dbd801e" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9c908063-6335-4571-b949-068e8b6f409f TargetServiceName=null EntryTimestamp=2022-04-26T06:25:49.361Z InvocationID=8f3c36e9-1e02-44bc-94b9-c632dec4a50e ErrorCode=500 owner of resource 88a8a792-7764-4286-b404-e7a2a15b87e4 is jh0003 2022-04-26T06:25:51.649Z [qtp2085002312-17] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "9c908063-6335-4571-b949-068e8b6f409f" INVARIANT_UUID = "b006ae6a-aad8-4be2-9315-2c995dbd801e" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9c908063-6335-4571-b949-068e8b6f409f TargetServiceName=null EntryTimestamp=2022-04-26T06:25:49.361Z InvocationID=8f3c36e9-1e02-44bc-94b9-c632dec4a50e ErrorCode=500 validate before certification change. resource name=VDU VirtualBlockStorage, oldState=NOT_CERTIFIED_CHECKIN, owner userId=jh0003 2022-04-26T06:25:51.650Z [qtp2085002312-17] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "9c908063-6335-4571-b949-068e8b6f409f" INVARIANT_UUID = "b006ae6a-aad8-4be2-9315-2c995dbd801e" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9c908063-6335-4571-b949-068e8b6f409f TargetServiceName=null EntryTimestamp=2022-04-26T06:25:49.361Z InvocationID=8f3c36e9-1e02-44bc-94b9-c632dec4a50e ErrorCode=500 start performing certification change for resource 88a8a792-7764-4286-b404-e7a2a15b87e4 2022-04-26T06:25:52.448Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9c908063-6335-4571-b949-068e8b6f409f" INVARIANT_UUID = "b006ae6a-aad8-4be2-9315-2c995dbd801e" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "44465c5d-d82b-406e-9f0c-9731e383f4d6.assettoscatemplate" CURR_ARTIFACT_UUID = "44465c5d-d82b-406e-9f0c-9731e383f4d6.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualblockstorage-template.yml,0,44465c5d-d82b-406e-9f0c-9731e383f4d6.assettoscatemplate,1,87539a54-2cb2-41e4-9ceb-b7b45752266e" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=df993512-adc6-4a74-923c-1a8ff74bb90a TargetServiceName=null EntryTimestamp=2022-04-26T06:25:49.361Z InvocationID=8f3c36e9-1e02-44bc-94b9-c632dec4a50e ErrorCode=500 Starting to create Csar for component VDU VirtualBlockStorage 2022-04-26T06:25:52.550Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b11c7f28-a9d6-4aab-9d98-ea0bd9a816c1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:25:52.550Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b11c7f28-a9d6-4aab-9d98-ea0bd9a816c1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@136a47ef 2022-04-26T06:25:52.551Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b11c7f28-a9d6-4aab-9d98-ea0bd9a816c1 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:25:52.551Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b11c7f28-a9d6-4aab-9d98-ea0bd9a816c1 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:25:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:25:52.686Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:25:52.686Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:25:52.726Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9c908063-6335-4571-b949-068e8b6f409f" INVARIANT_UUID = "b006ae6a-aad8-4be2-9315-2c995dbd801e" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "44465c5d-d82b-406e-9f0c-9731e383f4d6.assettoscatemplate" CURR_ARTIFACT_UUID = "44465c5d-d82b-406e-9f0c-9731e383f4d6.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualblockstorage-template.yml,0,44465c5d-d82b-406e-9f0c-9731e383f4d6.assettoscatemplate,1,87539a54-2cb2-41e4-9ceb-b7b45752266e" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=df993512-adc6-4a74-923c-1a8ff74bb90a TargetServiceName=null EntryTimestamp=2022-04-26T06:25:49.361Z InvocationID=8f3c36e9-1e02-44bc-94b9-c632dec4a50e ErrorCode=500 Ended create Csar for component VDU VirtualBlockStorage 2022-04-26T06:25:53.582Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=b11c7f28-a9d6-4aab-9d98-ea0bd9a816c1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:25:53.583Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=b11c7f28-a9d6-4aab-9d98-ea0bd9a816c1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:25:53.849Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "9c908063-6335-4571-b949-068e8b6f409f" INVARIANT_UUID = "b006ae6a-aad8-4be2-9315-2c995dbd801e" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=cbe0eb73-a97f-4588-b271-5d570d60cab9 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:49.361Z InvocationID=8f3c36e9-1e02-44bc-94b9-c632dec4a50e ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:53.850Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "9c908063-6335-4571-b949-068e8b6f409f" INVARIANT_UUID = "b006ae6a-aad8-4be2-9315-2c995dbd801e" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=cbe0eb73-a97f-4588-b271-5d570d60cab9 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:49.361Z InvocationID=8f3c36e9-1e02-44bc-94b9-c632dec4a50e ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:53.850Z [qtp2085002312-17] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "9c908063-6335-4571-b949-068e8b6f409f" INVARIANT_UUID = "b006ae6a-aad8-4be2-9315-2c995dbd801e" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=cbe0eb73-a97f-4588-b271-5d570d60cab9 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:49.361Z InvocationID=8f3c36e9-1e02-44bc-94b9-c632dec4a50e ErrorCode=500 unlock component 88a8a792-7764-4286-b404-e7a2a15b87e4 2022-04-26T06:25:53.850Z [qtp2085002312-17] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "9c908063-6335-4571-b949-068e8b6f409f" INVARIANT_UUID = "b006ae6a-aad8-4be2-9315-2c995dbd801e" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=cbe0eb73-a97f-4588-b271-5d570d60cab9 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:49.361Z InvocationID=8f3c36e9-1e02-44bc-94b9-c632dec4a50e ErrorCode=500 During change state, another component 88a8a792-7764-4286-b404-e7a2a15b87e4 has been created/updated 2022-04-26T06:25:53.861Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "9c908063-6335-4571-b949-068e8b6f409f" INVARIANT_UUID = "b006ae6a-aad8-4be2-9315-2c995dbd801e" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=4500 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=cbe0eb73-a97f-4588-b271-5d570d60cab9 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:49.361Z InvocationID=8f3c36e9-1e02-44bc-94b9-c632dec4a50e ErrorCode=500 LogTimestamp=2022-04-26T06:25:53.861073Z ResponseDesc=Created Exiting. 2022-04-26T06:25:53.862Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:25:53.943Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=941bc71c-ebb5-454a-87f7-e1af520f5951 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:53.943404Z InvocationID=2c5871cf-7db1-41c2-acb7-6fbabadaf819 LogTimestamp=2022-04-26T06:25:53.943485Z Entering 2022-04-26T06:25:53.944Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=941bc71c-ebb5-454a-87f7-e1af520f5951 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:53.943404Z InvocationID=2c5871cf-7db1-41c2-acb7-6fbabadaf819 LogTimestamp=2022-04-26T06:25:53.943485Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:25:53.954Z [JanusGraph Cluster-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:25:53.955Z [JanusGraph Cluster-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 16000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:25:54.255Z [cluster2-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:25:54.257Z [cluster2-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 30000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:25:54.944Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:53.944Z InvocationID=308f8a01-8ec1-4ae9-9c41-2ea4b0596eb6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 97287295-ddf7-4f36-aa8f-299f9c2b7846 2022-04-26T06:25:55.049Z [qtp2085002312-16] INFO o.o.s.b.c.l.LifecycleBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:53.944Z InvocationID=308f8a01-8ec1-4ae9-9c41-2ea4b0596eb6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb owner of resource 97287295-ddf7-4f36-aa8f-299f9c2b7846 is jh0003 2022-04-26T06:25:55.445Z [qtp2085002312-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "a83b1f98-3483-45e3-8b46-a2fb870c1458" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:53.944Z InvocationID=308f8a01-8ec1-4ae9-9c41-2ea4b0596eb6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:55.446Z [qtp2085002312-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "a83b1f98-3483-45e3-8b46-a2fb870c1458" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:25:53.944Z InvocationID=308f8a01-8ec1-4ae9-9c41-2ea4b0596eb6 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:55.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a73f68fa-a214-4aa7-aa42-29c69234ae54 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:25:55.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a73f68fa-a214-4aa7-aa42-29c69234ae54 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7532726e 2022-04-26T06:25:55.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a73f68fa-a214-4aa7-aa42-29c69234ae54 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:25:55.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a73f68fa-a214-4aa7-aa42-29c69234ae54 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:25:56.239Z [qtp2085002312-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "a83b1f98-3483-45e3-8b46-a2fb870c1458" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a6dc7914-3a9f-4cee-9cad-f028aafe1a5e TargetServiceName=null EntryTimestamp=2022-04-26T06:25:53.944Z InvocationID=308f8a01-8ec1-4ae9-9c41-2ea4b0596eb6 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:56.240Z [qtp2085002312-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "a83b1f98-3483-45e3-8b46-a2fb870c1458" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a6dc7914-3a9f-4cee-9cad-f028aafe1a5e TargetServiceName=null EntryTimestamp=2022-04-26T06:25:53.944Z InvocationID=308f8a01-8ec1-4ae9-9c41-2ea4b0596eb6 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:56.263Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a83b1f98-3483-45e3-8b46-a2fb870c1458" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VNF" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a6dc7914-3a9f-4cee-9cad-f028aafe1a5e TargetServiceName=null EntryTimestamp=2022-04-26T06:25:53.944Z InvocationID=308f8a01-8ec1-4ae9-9c41-2ea4b0596eb6 ErrorCode=500 lock resource with id 403dc22e-bea0-4af4-82f5-793ab90a1ab4 2022-04-26T06:25:56.265Z [qtp2085002312-16] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a83b1f98-3483-45e3-8b46-a2fb870c1458" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VNF" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a6dc7914-3a9f-4cee-9cad-f028aafe1a5e TargetServiceName=null EntryTimestamp=2022-04-26T06:25:53.944Z InvocationID=308f8a01-8ec1-4ae9-9c41-2ea4b0596eb6 ErrorCode=500 owner of resource 403dc22e-bea0-4af4-82f5-793ab90a1ab4 is jh0003 2022-04-26T06:25:56.687Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=a73f68fa-a214-4aa7-aa42-29c69234ae54 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:25:56.688Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=a73f68fa-a214-4aa7-aa42-29c69234ae54 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:25:57.455Z [qtp2085002312-16] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "a6dc7914-3a9f-4cee-9cad-f028aafe1a5e" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a6dc7914-3a9f-4cee-9cad-f028aafe1a5e TargetServiceName=null EntryTimestamp=2022-04-26T06:25:53.944Z InvocationID=308f8a01-8ec1-4ae9-9c41-2ea4b0596eb6 ErrorCode=500 owner of resource 403dc22e-bea0-4af4-82f5-793ab90a1ab4 is jh0003 2022-04-26T06:25:57.455Z [qtp2085002312-16] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "a6dc7914-3a9f-4cee-9cad-f028aafe1a5e" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a6dc7914-3a9f-4cee-9cad-f028aafe1a5e TargetServiceName=null EntryTimestamp=2022-04-26T06:25:53.944Z InvocationID=308f8a01-8ec1-4ae9-9c41-2ea4b0596eb6 ErrorCode=500 validate before certification change. resource name=VNF, oldState=NOT_CERTIFIED_CHECKIN, owner userId=jh0003 2022-04-26T06:25:57.455Z [qtp2085002312-16] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "a6dc7914-3a9f-4cee-9cad-f028aafe1a5e" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a6dc7914-3a9f-4cee-9cad-f028aafe1a5e TargetServiceName=null EntryTimestamp=2022-04-26T06:25:53.944Z InvocationID=308f8a01-8ec1-4ae9-9c41-2ea4b0596eb6 ErrorCode=500 start performing certification change for resource 403dc22e-bea0-4af4-82f5-793ab90a1ab4 2022-04-26T06:25:58.043Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a6dc7914-3a9f-4cee-9cad-f028aafe1a5e" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "97287295-ddf7-4f36-aa8f-299f9c2b7846.assettoscatemplate" CURR_ARTIFACT_UUID = "97287295-ddf7-4f36-aa8f-299f9c2b7846.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnf-template.yml,0,97287295-ddf7-4f36-aa8f-299f9c2b7846.assettoscatemplate,1,4316bcf0-bcaa-4ee9-a322-cc7dcc7cacd8" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=88c21622-772b-468a-b53b-32540c76cbb7 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:53.944Z InvocationID=308f8a01-8ec1-4ae9-9c41-2ea4b0596eb6 ErrorCode=500 Starting to create Csar for component VNF 2022-04-26T06:25:58.261Z [qtp2085002312-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a6dc7914-3a9f-4cee-9cad-f028aafe1a5e" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "97287295-ddf7-4f36-aa8f-299f9c2b7846.assettoscatemplate" CURR_ARTIFACT_UUID = "97287295-ddf7-4f36-aa8f-299f9c2b7846.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnf-template.yml,0,97287295-ddf7-4f36-aa8f-299f9c2b7846.assettoscatemplate,1,4316bcf0-bcaa-4ee9-a322-cc7dcc7cacd8" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=88c21622-772b-468a-b53b-32540c76cbb7 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:53.944Z InvocationID=308f8a01-8ec1-4ae9-9c41-2ea4b0596eb6 ErrorCode=500 Ended create Csar for component VNF 2022-04-26T06:25:58.440Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3beb52c-ef3c-448d-85c2-091dd60c6643 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:25:58.440Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3beb52c-ef3c-448d-85c2-091dd60c6643 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6dac8277 2022-04-26T06:25:58.440Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3beb52c-ef3c-448d-85c2-091dd60c6643 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:25:58.440Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3beb52c-ef3c-448d-85c2-091dd60c6643 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:25:59.465Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f3beb52c-ef3c-448d-85c2-091dd60c6643 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:25:59.466Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f3beb52c-ef3c-448d-85c2-091dd60c6643 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:25:59.847Z [qtp2085002312-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "a6dc7914-3a9f-4cee-9cad-f028aafe1a5e" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=31840976-6e9c-4541-8f19-16a932992799 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:53.944Z InvocationID=308f8a01-8ec1-4ae9-9c41-2ea4b0596eb6 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:25:59.848Z [qtp2085002312-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "a6dc7914-3a9f-4cee-9cad-f028aafe1a5e" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=31840976-6e9c-4541-8f19-16a932992799 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:53.944Z InvocationID=308f8a01-8ec1-4ae9-9c41-2ea4b0596eb6 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:25:59.848Z [qtp2085002312-16] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "a6dc7914-3a9f-4cee-9cad-f028aafe1a5e" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=31840976-6e9c-4541-8f19-16a932992799 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:53.944Z InvocationID=308f8a01-8ec1-4ae9-9c41-2ea4b0596eb6 ErrorCode=500 unlock component 403dc22e-bea0-4af4-82f5-793ab90a1ab4 2022-04-26T06:25:59.848Z [qtp2085002312-16] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "a6dc7914-3a9f-4cee-9cad-f028aafe1a5e" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=31840976-6e9c-4541-8f19-16a932992799 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:53.944Z InvocationID=308f8a01-8ec1-4ae9-9c41-2ea4b0596eb6 ErrorCode=500 During change state, another component 403dc22e-bea0-4af4-82f5-793ab90a1ab4 has been created/updated 2022-04-26T06:25:59.860Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "a6dc7914-3a9f-4cee-9cad-f028aafe1a5e" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5915 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=31840976-6e9c-4541-8f19-16a932992799 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:53.944Z InvocationID=308f8a01-8ec1-4ae9-9c41-2ea4b0596eb6 ErrorCode=500 LogTimestamp=2022-04-26T06:25:59.859750Z ResponseDesc=Created Exiting. 2022-04-26T06:25:59.861Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:25:59.949Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ca7d5f02-9db9-4971-9d5e-dbb77d09c4da InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:59.948632Z InvocationID=f8483f39-9911-4825-a177-dfa8651f5304 LogTimestamp=2022-04-26T06:25:59.948713Z Entering 2022-04-26T06:25:59.949Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ca7d5f02-9db9-4971-9d5e-dbb77d09c4da InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:25:59.948632Z InvocationID=f8483f39-9911-4825-a177-dfa8651f5304 LogTimestamp=2022-04-26T06:25:59.948713Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:26:01.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4137afbc-f901-4be2-bd19-62d54db4d523 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:26:01.531Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4137afbc-f901-4be2-bd19-62d54db4d523 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@17ad3107 2022-04-26T06:26:01.531Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4137afbc-f901-4be2-bd19-62d54db4d523 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:26:01.531Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4137afbc-f901-4be2-bd19-62d54db4d523 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:26:01.557Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:59.950Z InvocationID=c226693b-b757-4956-9fcb-2f7e09703ea8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 36b3fccc-7cea-4cd3-b2e9-6d11cb572122 2022-04-26T06:26:01.864Z [qtp2085002312-18] INFO o.o.s.b.c.l.LifecycleBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:59.950Z InvocationID=c226693b-b757-4956-9fcb-2f7e09703ea8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb owner of resource 36b3fccc-7cea-4cd3-b2e9-6d11cb572122 is jh0003 2022-04-26T06:26:02.441Z [qtp2085002312-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "NS" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "e6b82461-810c-44c3-b8ca-965126f8a19d" INVARIANT_UUID = "d072405b-9c53-4793-9678-68d02bed3d8a" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:25:59.950Z InvocationID=c226693b-b757-4956-9fcb-2f7e09703ea8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:26:02.441Z [qtp2085002312-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "NS" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "e6b82461-810c-44c3-b8ca-965126f8a19d" INVARIANT_UUID = "d072405b-9c53-4793-9678-68d02bed3d8a" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:25:59.950Z InvocationID=c226693b-b757-4956-9fcb-2f7e09703ea8 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:26:02.575Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=4137afbc-f901-4be2-bd19-62d54db4d523 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:26:02.576Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=4137afbc-f901-4be2-bd19-62d54db4d523 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:26:03.832Z [qtp2085002312-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "NS" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "e6b82461-810c-44c3-b8ca-965126f8a19d" INVARIANT_UUID = "d072405b-9c53-4793-9678-68d02bed3d8a" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8f1de083-7221-4189-8616-926206de67a0 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:59.950Z InvocationID=c226693b-b757-4956-9fcb-2f7e09703ea8 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:26:03.833Z [qtp2085002312-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "NS" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "e6b82461-810c-44c3-b8ca-965126f8a19d" INVARIANT_UUID = "d072405b-9c53-4793-9678-68d02bed3d8a" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8f1de083-7221-4189-8616-926206de67a0 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:59.950Z InvocationID=c226693b-b757-4956-9fcb-2f7e09703ea8 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:26:03.932Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e6b82461-810c-44c3-b8ca-965126f8a19d" INVARIANT_UUID = "d072405b-9c53-4793-9678-68d02bed3d8a" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NS" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8f1de083-7221-4189-8616-926206de67a0 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:59.950Z InvocationID=c226693b-b757-4956-9fcb-2f7e09703ea8 ErrorCode=500 lock resource with id e21a85c7-1f89-4270-9d7d-9c86c43cd928 2022-04-26T06:26:03.933Z [qtp2085002312-18] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e6b82461-810c-44c3-b8ca-965126f8a19d" INVARIANT_UUID = "d072405b-9c53-4793-9678-68d02bed3d8a" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NS" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8f1de083-7221-4189-8616-926206de67a0 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:59.950Z InvocationID=c226693b-b757-4956-9fcb-2f7e09703ea8 ErrorCode=500 owner of resource e21a85c7-1f89-4270-9d7d-9c86c43cd928 is jh0003 2022-04-26T06:26:04.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9caaa1db-4414-4a3b-9d14-0b9cf5340f9d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:26:04.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9caaa1db-4414-4a3b-9d14-0b9cf5340f9d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@75a19560 2022-04-26T06:26:04.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9caaa1db-4414-4a3b-9d14-0b9cf5340f9d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:26:04.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9caaa1db-4414-4a3b-9d14-0b9cf5340f9d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:26:05.368Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:26:05.378Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:26:05.677Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=9caaa1db-4414-4a3b-9d14-0b9cf5340f9d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:26:05.678Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=9caaa1db-4414-4a3b-9d14-0b9cf5340f9d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:26:06.247Z [qtp2085002312-18] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "NS" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "8f1de083-7221-4189-8616-926206de67a0" INVARIANT_UUID = "d072405b-9c53-4793-9678-68d02bed3d8a" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8f1de083-7221-4189-8616-926206de67a0 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:59.950Z InvocationID=c226693b-b757-4956-9fcb-2f7e09703ea8 ErrorCode=500 owner of resource e21a85c7-1f89-4270-9d7d-9c86c43cd928 is jh0003 2022-04-26T06:26:06.247Z [qtp2085002312-18] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "NS" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "8f1de083-7221-4189-8616-926206de67a0" INVARIANT_UUID = "d072405b-9c53-4793-9678-68d02bed3d8a" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8f1de083-7221-4189-8616-926206de67a0 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:59.950Z InvocationID=c226693b-b757-4956-9fcb-2f7e09703ea8 ErrorCode=500 validate before certification change. resource name=NS, oldState=NOT_CERTIFIED_CHECKIN, owner userId=jh0003 2022-04-26T06:26:06.247Z [qtp2085002312-18] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "NS" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "8f1de083-7221-4189-8616-926206de67a0" INVARIANT_UUID = "d072405b-9c53-4793-9678-68d02bed3d8a" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8f1de083-7221-4189-8616-926206de67a0 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:59.950Z InvocationID=c226693b-b757-4956-9fcb-2f7e09703ea8 ErrorCode=500 start performing certification change for resource e21a85c7-1f89-4270-9d7d-9c86c43cd928 2022-04-26T06:26:06.359Z [cluster1-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:26:06.360Z [cluster1-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 32000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:26:07.429Z [qtp2085002312-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "8f1de083-7221-4189-8616-926206de67a0" INVARIANT_UUID = "d072405b-9c53-4793-9678-68d02bed3d8a" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "36b3fccc-7cea-4cd3-b2e9-6d11cb572122.assettoscatemplate" CURR_ARTIFACT_UUID = "36b3fccc-7cea-4cd3-b2e9-6d11cb572122.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ns-template.yml,0,36b3fccc-7cea-4cd3-b2e9-6d11cb572122.assettoscatemplate,1,e8e11789-ad93-488c-a85c-e58fa70995c8" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=88abd8b3-f33c-4f2a-b89f-9cf917318dd1 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:59.950Z InvocationID=c226693b-b757-4956-9fcb-2f7e09703ea8 ErrorCode=500 Starting to create Csar for component NS 2022-04-26T06:26:07.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0adad8d-3ff4-41a8-a8a0-4fae617755b4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:26:07.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0adad8d-3ff4-41a8-a8a0-4fae617755b4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2e5799b1 2022-04-26T06:26:07.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0adad8d-3ff4-41a8-a8a0-4fae617755b4 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:26:07.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0adad8d-3ff4-41a8-a8a0-4fae617755b4 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:26:07.659Z [qtp2085002312-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "8f1de083-7221-4189-8616-926206de67a0" INVARIANT_UUID = "d072405b-9c53-4793-9678-68d02bed3d8a" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "36b3fccc-7cea-4cd3-b2e9-6d11cb572122.assettoscatemplate" CURR_ARTIFACT_UUID = "36b3fccc-7cea-4cd3-b2e9-6d11cb572122.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ns-template.yml,0,36b3fccc-7cea-4cd3-b2e9-6d11cb572122.assettoscatemplate,1,e8e11789-ad93-488c-a85c-e58fa70995c8" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=88abd8b3-f33c-4f2a-b89f-9cf917318dd1 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:59.950Z InvocationID=c226693b-b757-4956-9fcb-2f7e09703ea8 ErrorCode=500 Ended create Csar for component NS 2022-04-26T06:26:07.676Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:26:08.653Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f0adad8d-3ff4-41a8-a8a0-4fae617755b4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:26:08.653Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f0adad8d-3ff4-41a8-a8a0-4fae617755b4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:26:09.047Z [qtp2085002312-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "NS" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "8f1de083-7221-4189-8616-926206de67a0" INVARIANT_UUID = "d072405b-9c53-4793-9678-68d02bed3d8a" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9e1ea1ab-ac68-4afb-ac6c-55af53313f47 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:59.950Z InvocationID=c226693b-b757-4956-9fcb-2f7e09703ea8 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:26:09.048Z [qtp2085002312-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "NS" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "8f1de083-7221-4189-8616-926206de67a0" INVARIANT_UUID = "d072405b-9c53-4793-9678-68d02bed3d8a" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9e1ea1ab-ac68-4afb-ac6c-55af53313f47 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:59.950Z InvocationID=c226693b-b757-4956-9fcb-2f7e09703ea8 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:26:09.048Z [qtp2085002312-18] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "NS" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "8f1de083-7221-4189-8616-926206de67a0" INVARIANT_UUID = "d072405b-9c53-4793-9678-68d02bed3d8a" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9e1ea1ab-ac68-4afb-ac6c-55af53313f47 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:59.950Z InvocationID=c226693b-b757-4956-9fcb-2f7e09703ea8 ErrorCode=500 unlock component e21a85c7-1f89-4270-9d7d-9c86c43cd928 2022-04-26T06:26:09.049Z [qtp2085002312-18] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "NS" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "8f1de083-7221-4189-8616-926206de67a0" INVARIANT_UUID = "d072405b-9c53-4793-9678-68d02bed3d8a" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9e1ea1ab-ac68-4afb-ac6c-55af53313f47 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:59.950Z InvocationID=c226693b-b757-4956-9fcb-2f7e09703ea8 ErrorCode=500 During change state, another component e21a85c7-1f89-4270-9d7d-9c86c43cd928 has been created/updated 2022-04-26T06:26:09.059Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "NS" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "8f1de083-7221-4189-8616-926206de67a0" INVARIANT_UUID = "d072405b-9c53-4793-9678-68d02bed3d8a" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=9109 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9e1ea1ab-ac68-4afb-ac6c-55af53313f47 TargetServiceName=null EntryTimestamp=2022-04-26T06:25:59.950Z InvocationID=c226693b-b757-4956-9fcb-2f7e09703ea8 ErrorCode=500 LogTimestamp=2022-04-26T06:26:09.059099Z ResponseDesc=Created Exiting. 2022-04-26T06:26:09.060Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:26:09.131Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=2380ac7f-68ca-4c9b-9cae-b9b56995686f InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:09.131383Z InvocationID=1e9aefd7-6b60-4627-9fae-cd2796177685 LogTimestamp=2022-04-26T06:26:09.131520Z Entering 2022-04-26T06:26:09.132Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=2380ac7f-68ca-4c9b-9cae-b9b56995686f InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:09.131383Z InvocationID=1e9aefd7-6b60-4627-9fae-cd2796177685 LogTimestamp=2022-04-26T06:26:09.131520Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:26:10.238Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:26:09.132Z InvocationID=f8a55bbc-f18f-49c1-a9f2-162549076425 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id 1688dc1a-b973-40f8-a018-27ec14af3322 2022-04-26T06:26:10.339Z [qtp2085002312-17] INFO o.o.s.b.c.l.LifecycleBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:26:09.132Z InvocationID=f8a55bbc-f18f-49c1-a9f2-162549076425 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb owner of resource 1688dc1a-b973-40f8-a018-27ec14af3322 is jh0003 2022-04-26T06:26:10.547Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=778fa27e-fb40-4213-8e11-fda08c1500ce PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:26:10.548Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=778fa27e-fb40-4213-8e11-fda08c1500ce PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@138a1399 2022-04-26T06:26:10.548Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=778fa27e-fb40-4213-8e11-fda08c1500ce PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:26:10.548Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=778fa27e-fb40-4213-8e11-fda08c1500ce PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:26:10.658Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d0a8b647-2ab6-4218-abc2-d3212e6800c2" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "2.0" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:26:09.132Z InvocationID=f8a55bbc-f18f-49c1-a9f2-162549076425 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:26:10.659Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d0a8b647-2ab6-4218-abc2-d3212e6800c2" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "2.0" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:26:09.132Z InvocationID=f8a55bbc-f18f-49c1-a9f2-162549076425 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:26:10.980Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d0a8b647-2ab6-4218-abc2-d3212e6800c2" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "2.0" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d5f1539c-1c39-499f-80ef-b63c230c9a76 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:09.132Z InvocationID=f8a55bbc-f18f-49c1-a9f2-162549076425 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:26:10.981Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d0a8b647-2ab6-4218-abc2-d3212e6800c2" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "2.0" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d5f1539c-1c39-499f-80ef-b63c230c9a76 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:09.132Z InvocationID=f8a55bbc-f18f-49c1-a9f2-162549076425 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:26:10.998Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d0a8b647-2ab6-4218-abc2-d3212e6800c2" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "2.1" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VNF" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d5f1539c-1c39-499f-80ef-b63c230c9a76 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:09.132Z InvocationID=f8a55bbc-f18f-49c1-a9f2-162549076425 ErrorCode=500 lock resource with id a70cd4bc-100a-4452-b728-fe1d400601b5 2022-04-26T06:26:11.000Z [qtp2085002312-17] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d0a8b647-2ab6-4218-abc2-d3212e6800c2" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "2.1" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VNF" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d5f1539c-1c39-499f-80ef-b63c230c9a76 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:09.132Z InvocationID=f8a55bbc-f18f-49c1-a9f2-162549076425 ErrorCode=500 owner of resource a70cd4bc-100a-4452-b728-fe1d400601b5 is jh0003 2022-04-26T06:26:11.562Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=778fa27e-fb40-4213-8e11-fda08c1500ce ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:26:11.563Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=778fa27e-fb40-4213-8e11-fda08c1500ce ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:26:11.650Z [qtp2085002312-17] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d5f1539c-1c39-499f-80ef-b63c230c9a76" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "2.1" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d5f1539c-1c39-499f-80ef-b63c230c9a76 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:09.132Z InvocationID=f8a55bbc-f18f-49c1-a9f2-162549076425 ErrorCode=500 owner of resource a70cd4bc-100a-4452-b728-fe1d400601b5 is jh0003 2022-04-26T06:26:11.650Z [qtp2085002312-17] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d5f1539c-1c39-499f-80ef-b63c230c9a76" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "2.1" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d5f1539c-1c39-499f-80ef-b63c230c9a76 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:09.132Z InvocationID=f8a55bbc-f18f-49c1-a9f2-162549076425 ErrorCode=500 validate before certification change. resource name=VNF, oldState=NOT_CERTIFIED_CHECKIN, owner userId=jh0003 2022-04-26T06:26:11.650Z [qtp2085002312-17] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d5f1539c-1c39-499f-80ef-b63c230c9a76" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "2.1" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d5f1539c-1c39-499f-80ef-b63c230c9a76 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:09.132Z InvocationID=f8a55bbc-f18f-49c1-a9f2-162549076425 ErrorCode=500 start performing certification change for resource a70cd4bc-100a-4452-b728-fe1d400601b5 2022-04-26T06:26:11.744Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d5f1539c-1c39-499f-80ef-b63c230c9a76" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "" CURR_VERSION = "2.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "403dc22e-bea0-4af4-82f5-793ab90a1ab4.assettoscatemplate" CURR_ARTIFACT_UUID = "403dc22e-bea0-4af4-82f5-793ab90a1ab4.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnf-template.yml,0,403dc22e-bea0-4af4-82f5-793ab90a1ab4.assettoscatemplate,2,88c21622-772b-468a-b53b-32540c76cbb7" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=804f37cd-9b4f-4e91-a559-80878ca76b5b TargetServiceName=null EntryTimestamp=2022-04-26T06:26:09.132Z InvocationID=f8a55bbc-f18f-49c1-a9f2-162549076425 ErrorCode=500 Starting to create Csar for component VNF 2022-04-26T06:26:11.981Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d5f1539c-1c39-499f-80ef-b63c230c9a76" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "" CURR_VERSION = "2.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "403dc22e-bea0-4af4-82f5-793ab90a1ab4.assettoscatemplate" CURR_ARTIFACT_UUID = "403dc22e-bea0-4af4-82f5-793ab90a1ab4.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnf-template.yml,0,403dc22e-bea0-4af4-82f5-793ab90a1ab4.assettoscatemplate,2,88c21622-772b-468a-b53b-32540c76cbb7" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=804f37cd-9b4f-4e91-a559-80878ca76b5b TargetServiceName=null EntryTimestamp=2022-04-26T06:26:09.132Z InvocationID=f8a55bbc-f18f-49c1-a9f2-162549076425 ErrorCode=500 Ended create Csar for component VNF 2022-04-26T06:26:12.594Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d5f1539c-1c39-499f-80ef-b63c230c9a76" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "2.1" CURR_VERSION = "3.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=de4538f8-130a-4333-9507-d6b139674c5a TargetServiceName=null EntryTimestamp=2022-04-26T06:26:09.132Z InvocationID=f8a55bbc-f18f-49c1-a9f2-162549076425 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:26:12.595Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d5f1539c-1c39-499f-80ef-b63c230c9a76" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "2.1" CURR_VERSION = "3.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=de4538f8-130a-4333-9507-d6b139674c5a TargetServiceName=null EntryTimestamp=2022-04-26T06:26:09.132Z InvocationID=f8a55bbc-f18f-49c1-a9f2-162549076425 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:26:12.595Z [qtp2085002312-17] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d5f1539c-1c39-499f-80ef-b63c230c9a76" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "2.1" CURR_VERSION = "3.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=de4538f8-130a-4333-9507-d6b139674c5a TargetServiceName=null EntryTimestamp=2022-04-26T06:26:09.132Z InvocationID=f8a55bbc-f18f-49c1-a9f2-162549076425 ErrorCode=500 unlock component a70cd4bc-100a-4452-b728-fe1d400601b5 2022-04-26T06:26:12.595Z [qtp2085002312-17] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d5f1539c-1c39-499f-80ef-b63c230c9a76" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "2.1" CURR_VERSION = "3.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=de4538f8-130a-4333-9507-d6b139674c5a TargetServiceName=null EntryTimestamp=2022-04-26T06:26:09.132Z InvocationID=f8a55bbc-f18f-49c1-a9f2-162549076425 ErrorCode=500 During change state, another component a70cd4bc-100a-4452-b728-fe1d400601b5 has been created/updated 2022-04-26T06:26:12.605Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d5f1539c-1c39-499f-80ef-b63c230c9a76" INVARIANT_UUID = "a65a6432-4ed2-4e08-abf0-7b35e693eac7" PREV_VERSION = "2.1" CURR_VERSION = "3.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=3472 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=de4538f8-130a-4333-9507-d6b139674c5a TargetServiceName=null EntryTimestamp=2022-04-26T06:26:09.132Z InvocationID=f8a55bbc-f18f-49c1-a9f2-162549076425 ErrorCode=500 LogTimestamp=2022-04-26T06:26:12.604699Z ResponseDesc=Created Exiting. 2022-04-26T06:26:12.605Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:26:12.617Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=d6024b0d-c626-4bc9-beaf-e7e5474d84c3 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:12.616847Z InvocationID=509b0549-78ee-4d32-afa8-70c4c097fb7c LogTimestamp=2022-04-26T06:26:12.616950Z Entering 2022-04-26T06:26:12.617Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=d6024b0d-c626-4bc9-beaf-e7e5474d84c3 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:12.616847Z InvocationID=509b0549-78ee-4d32-afa8-70c4c097fb7c LogTimestamp=2022-04-26T06:26:12.616950Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:26:12.794Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:26:12.618Z InvocationID=a6823818-2262-44cc-8c8c-5430093481b0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:26:12.794Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:26:12.618Z InvocationID=a6823818-2262-44cc-8c8c-5430093481b0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:26:12.795Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:26:12.618Z InvocationID=a6823818-2262-44cc-8c8c-5430093481b0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:26:12.795Z [qtp2085002312-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:26:12.618Z InvocationID=a6823818-2262-44cc-8c8c-5430093481b0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:26:12.840Z [qtp2085002312-14] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Create Resource - validateCapabilityTypesCreate ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:26:12.618Z InvocationID=a6823818-2262-44cc-8c8c-5430093481b0 ErrorCode=400 Capability Type tosca.capabilities.nfv.ContainerGroupable required is missing 2022-04-26T06:26:12.841Z [qtp2085002312-14] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Create Resource - validateCapabilityTypesCreate ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:26:12.618Z InvocationID=a6823818-2262-44cc-8c8c-5430093481b0 ErrorCode=500 Operation towards database failed. 2022-04-26T06:26:12.844Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "osContainer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "" INVARIANT_UUID = "" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "400" DESC = "SVC4115: Error: Invalid Content. Missing Capability Type 'tosca.capabilities.nfv.ContainerGroupable'." TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.osContainer" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Create Resource - validateCapabilityTypesCreate ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:26:12.618Z InvocationID=a6823818-2262-44cc-8c8c-5430093481b0 ErrorCode=500 lock resource with name Oscontainer 2022-04-26T06:26:12.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=edd33f08-78df-4619-8893-78841a182660 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:26:12.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=edd33f08-78df-4619-8893-78841a182660 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@344620f0 2022-04-26T06:26:12.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=edd33f08-78df-4619-8893-78841a182660 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:26:12.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=edd33f08-78df-4619-8893-78841a182660 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:26:12.886Z [qtp2085002312-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "osContainer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "db936b3f-21dd-4f04-96f6-993e778f5cad" INVARIANT_UUID = "42ebac09-c939-40ab-b76d-a97871ad871b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.osContainer" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Create Resource - validateCapabilityTypesCreate ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=db936b3f-21dd-4f04-96f6-993e778f5cad TargetServiceName=null EntryTimestamp=2022-04-26T06:26:12.618Z InvocationID=a6823818-2262-44cc-8c8c-5430093481b0 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:26:12.887Z [qtp2085002312-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "osContainer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "db936b3f-21dd-4f04-96f6-993e778f5cad" INVARIANT_UUID = "42ebac09-c939-40ab-b76d-a97871ad871b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.osContainer" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=db936b3f-21dd-4f04-96f6-993e778f5cad TargetServiceName=null EntryTimestamp=2022-04-26T06:26:12.618Z InvocationID=a6823818-2262-44cc-8c8c-5430093481b0 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:26:12.986Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "osContainer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "db936b3f-21dd-4f04-96f6-993e778f5cad" INVARIANT_UUID = "42ebac09-c939-40ab-b76d-a97871ad871b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.osContainer" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=db936b3f-21dd-4f04-96f6-993e778f5cad TargetServiceName=null EntryTimestamp=2022-04-26T06:26:12.618Z InvocationID=a6823818-2262-44cc-8c8c-5430093481b0 ErrorCode=500 lock resource with id 889bf5a2-1d37-4af0-bc68-cbfe84f8f30a 2022-04-26T06:26:13.584Z [qtp2085002312-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "osContainer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "db936b3f-21dd-4f04-96f6-993e778f5cad" INVARIANT_UUID = "42ebac09-c939-40ab-b76d-a97871ad871b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "889bf5a2-1d37-4af0-bc68-cbfe84f8f30a.assettoscatemplate" CURR_ARTIFACT_UUID = "889bf5a2-1d37-4af0-bc68-cbfe84f8f30a.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Oscontainer-template.yml,0,889bf5a2-1d37-4af0-bc68-cbfe84f8f30a.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d9275a3c-86cb-4fc7-bbfe-01207a1c9aa4 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:12.618Z InvocationID=a6823818-2262-44cc-8c8c-5430093481b0 ErrorCode=500 Starting to create Csar for component osContainer 2022-04-26T06:26:13.777Z [qtp2085002312-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "osContainer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "db936b3f-21dd-4f04-96f6-993e778f5cad" INVARIANT_UUID = "42ebac09-c939-40ab-b76d-a97871ad871b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "889bf5a2-1d37-4af0-bc68-cbfe84f8f30a.assettoscatemplate" CURR_ARTIFACT_UUID = "889bf5a2-1d37-4af0-bc68-cbfe84f8f30a.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Oscontainer-template.yml,0,889bf5a2-1d37-4af0-bc68-cbfe84f8f30a.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d9275a3c-86cb-4fc7-bbfe-01207a1c9aa4 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:12.618Z InvocationID=a6823818-2262-44cc-8c8c-5430093481b0 ErrorCode=500 Ended create Csar for component osContainer 2022-04-26T06:26:13.778Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "osContainer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "db936b3f-21dd-4f04-96f6-993e778f5cad" INVARIANT_UUID = "42ebac09-c939-40ab-b76d-a97871ad871b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "889bf5a2-1d37-4af0-bc68-cbfe84f8f30a.assettoscatemplate" CURR_ARTIFACT_UUID = "889bf5a2-1d37-4af0-bc68-cbfe84f8f30a.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Oscontainer-template.yml,0,889bf5a2-1d37-4af0-bc68-cbfe84f8f30a.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d9275a3c-86cb-4fc7-bbfe-01207a1c9aa4 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:12.618Z InvocationID=a6823818-2262-44cc-8c8c-5430093481b0 ErrorCode=500 lock resource with id 889bf5a2-1d37-4af0-bc68-cbfe84f8f30a 2022-04-26T06:26:13.897Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=edd33f08-78df-4619-8893-78841a182660 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:26:13.897Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=edd33f08-78df-4619-8893-78841a182660 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:26:14.358Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "osContainer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "db936b3f-21dd-4f04-96f6-993e778f5cad" INVARIANT_UUID = "42ebac09-c939-40ab-b76d-a97871ad871b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "889bf5a2-1d37-4af0-bc68-cbfe84f8f30a.assettoscacsar" CURR_ARTIFACT_UUID = "889bf5a2-1d37-4af0-bc68-cbfe84f8f30a.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Oscontainer-csar.csar,0,889bf5a2-1d37-4af0-bc68-cbfe84f8f30a.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1740 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=87995962-dca8-45d2-9c82-2e01773ade46 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:12.618Z InvocationID=a6823818-2262-44cc-8c8c-5430093481b0 ErrorCode=500 LogTimestamp=2022-04-26T06:26:14.358456Z ResponseDesc=Created Exiting. 2022-04-26T06:26:14.359Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:26:14.378Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=fe91c365-27f0-433a-8a41-8141683b04fb InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:14.378061Z InvocationID=034ece58-e158-4e19-88ab-8bff042c3487 LogTimestamp=2022-04-26T06:26:14.378134Z Entering 2022-04-26T06:26:14.378Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=fe91c365-27f0-433a-8a41-8141683b04fb InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:14.378061Z InvocationID=034ece58-e158-4e19-88ab-8bff042c3487 LogTimestamp=2022-04-26T06:26:14.378134Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:26:14.479Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:26:14.379Z InvocationID=eef9d502-9384-4001-9a53-646fd6f0cd2f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:26:14.479Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:26:14.379Z InvocationID=eef9d502-9384-4001-9a53-646fd6f0cd2f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:26:14.479Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:26:14.379Z InvocationID=eef9d502-9384-4001-9a53-646fd6f0cd2f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:26:14.479Z [qtp2085002312-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:26:14.379Z InvocationID=eef9d502-9384-4001-9a53-646fd6f0cd2f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:26:14.488Z [qtp2085002312-18] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Create Resource - validateCapabilityTypesCreate ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:26:14.379Z InvocationID=eef9d502-9384-4001-9a53-646fd6f0cd2f ErrorCode=400 Capability Type tosca.capabilities.nfv.ContainerGroupable required is missing 2022-04-26T06:26:14.488Z [qtp2085002312-18] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Create Resource - validateCapabilityTypesCreate ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:26:14.379Z InvocationID=eef9d502-9384-4001-9a53-646fd6f0cd2f ErrorCode=500 Operation towards database failed. 2022-04-26T06:26:14.491Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "osContainerGroup" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "" INVARIANT_UUID = "" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "400" DESC = "SVC4115: Error: Invalid Content. Missing Capability Type 'tosca.capabilities.nfv.ContainerGroupable'." TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.osContainerGroup" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Create Resource - validateCapabilityTypesCreate ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:26:14.379Z InvocationID=eef9d502-9384-4001-9a53-646fd6f0cd2f ErrorCode=500 lock resource with name Oscontainergroup 2022-04-26T06:26:14.528Z [qtp2085002312-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "osContainerGroup" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "dba20e2e-b9b3-42d3-828e-df5b15580d14" INVARIANT_UUID = "4c890947-b485-4b3b-8757-038bb5dfd4ce" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.osContainerGroup" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Create Resource - validateCapabilityTypesCreate ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=dba20e2e-b9b3-42d3-828e-df5b15580d14 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:14.379Z InvocationID=eef9d502-9384-4001-9a53-646fd6f0cd2f ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:26:14.529Z [qtp2085002312-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "osContainerGroup" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "dba20e2e-b9b3-42d3-828e-df5b15580d14" INVARIANT_UUID = "4c890947-b485-4b3b-8757-038bb5dfd4ce" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.osContainerGroup" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=dba20e2e-b9b3-42d3-828e-df5b15580d14 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:14.379Z InvocationID=eef9d502-9384-4001-9a53-646fd6f0cd2f ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:26:14.571Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "osContainerGroup" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "dba20e2e-b9b3-42d3-828e-df5b15580d14" INVARIANT_UUID = "4c890947-b485-4b3b-8757-038bb5dfd4ce" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.osContainerGroup" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=dba20e2e-b9b3-42d3-828e-df5b15580d14 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:14.379Z InvocationID=eef9d502-9384-4001-9a53-646fd6f0cd2f ErrorCode=500 lock resource with id 326b9eec-3406-4881-916c-d2ba7fe4653d 2022-04-26T06:26:15.160Z [qtp2085002312-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "osContainerGroup" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "dba20e2e-b9b3-42d3-828e-df5b15580d14" INVARIANT_UUID = "4c890947-b485-4b3b-8757-038bb5dfd4ce" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "326b9eec-3406-4881-916c-d2ba7fe4653d.assettoscatemplate" CURR_ARTIFACT_UUID = "326b9eec-3406-4881-916c-d2ba7fe4653d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Oscontainergroup-template.yml,0,326b9eec-3406-4881-916c-d2ba7fe4653d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=14d7a5b8-3329-4d31-b44e-9cf150d168d4 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:14.379Z InvocationID=eef9d502-9384-4001-9a53-646fd6f0cd2f ErrorCode=500 Starting to create Csar for component osContainerGroup 2022-04-26T06:26:15.368Z [qtp2085002312-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "osContainerGroup" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "dba20e2e-b9b3-42d3-828e-df5b15580d14" INVARIANT_UUID = "4c890947-b485-4b3b-8757-038bb5dfd4ce" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "326b9eec-3406-4881-916c-d2ba7fe4653d.assettoscatemplate" CURR_ARTIFACT_UUID = "326b9eec-3406-4881-916c-d2ba7fe4653d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Oscontainergroup-template.yml,0,326b9eec-3406-4881-916c-d2ba7fe4653d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=14d7a5b8-3329-4d31-b44e-9cf150d168d4 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:14.379Z InvocationID=eef9d502-9384-4001-9a53-646fd6f0cd2f ErrorCode=500 Ended create Csar for component osContainerGroup 2022-04-26T06:26:15.369Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "osContainerGroup" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "dba20e2e-b9b3-42d3-828e-df5b15580d14" INVARIANT_UUID = "4c890947-b485-4b3b-8757-038bb5dfd4ce" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "326b9eec-3406-4881-916c-d2ba7fe4653d.assettoscatemplate" CURR_ARTIFACT_UUID = "326b9eec-3406-4881-916c-d2ba7fe4653d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Oscontainergroup-template.yml,0,326b9eec-3406-4881-916c-d2ba7fe4653d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=14d7a5b8-3329-4d31-b44e-9cf150d168d4 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:14.379Z InvocationID=eef9d502-9384-4001-9a53-646fd6f0cd2f ErrorCode=500 lock resource with id 326b9eec-3406-4881-916c-d2ba7fe4653d 2022-04-26T06:26:15.785Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7bbf7ae-4445-44c0-a5ea-6a16b923a12c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:26:15.785Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7bbf7ae-4445-44c0-a5ea-6a16b923a12c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@51c59042 2022-04-26T06:26:15.786Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7bbf7ae-4445-44c0-a5ea-6a16b923a12c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:26:15.786Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7bbf7ae-4445-44c0-a5ea-6a16b923a12c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:26:15.932Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "osContainerGroup" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "dba20e2e-b9b3-42d3-828e-df5b15580d14" INVARIANT_UUID = "4c890947-b485-4b3b-8757-038bb5dfd4ce" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "326b9eec-3406-4881-916c-d2ba7fe4653d.assettoscacsar" CURR_ARTIFACT_UUID = "326b9eec-3406-4881-916c-d2ba7fe4653d.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Oscontainergroup-csar.csar,0,326b9eec-3406-4881-916c-d2ba7fe4653d.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1552 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=21582497-6fb9-4f57-a83a-a4460ebf6d4b TargetServiceName=null EntryTimestamp=2022-04-26T06:26:14.379Z InvocationID=eef9d502-9384-4001-9a53-646fd6f0cd2f ErrorCode=500 LogTimestamp=2022-04-26T06:26:15.931839Z ResponseDesc=Created Exiting. 2022-04-26T06:26:15.932Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:26:15.945Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=4fde030a-cff0-4641-a1fd-dfcb7545e266 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:15.944785Z InvocationID=87232ba0-bd1b-41b6-ad2f-53b9d06f3689 LogTimestamp=2022-04-26T06:26:15.944825Z Entering 2022-04-26T06:26:15.945Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=4fde030a-cff0-4641-a1fd-dfcb7545e266 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:15.944785Z InvocationID=87232ba0-bd1b-41b6-ad2f-53b9d06f3689 LogTimestamp=2022-04-26T06:26:15.944825Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:26:16.084Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:26:15.945Z InvocationID=e52f6110-9c64-460c-ad46-c58c193e0d7f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id e0bf6d87-bebe-425f-9326-2dfd482d4d85 2022-04-26T06:26:16.097Z [qtp2085002312-14] INFO o.o.s.b.c.l.LifecycleBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:26:15.945Z InvocationID=e52f6110-9c64-460c-ad46-c58c193e0d7f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb owner of resource e0bf6d87-bebe-425f-9326-2dfd482d4d85 is jh0003 2022-04-26T06:26:16.141Z [qtp2085002312-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "895f054e-c898-492f-8100-718cf1b17db6" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "2.0" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:26:15.945Z InvocationID=e52f6110-9c64-460c-ad46-c58c193e0d7f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:26:16.141Z [qtp2085002312-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "895f054e-c898-492f-8100-718cf1b17db6" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "2.0" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2022-04-26T06:26:15.945Z InvocationID=e52f6110-9c64-460c-ad46-c58c193e0d7f ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:26:16.162Z [JanusGraph Cluster-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:26:16.163Z [JanusGraph Cluster-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 32000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:26:16.250Z [qtp2085002312-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "895f054e-c898-492f-8100-718cf1b17db6" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "2.0" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=698914d1-40a9-4b00-80ae-57904e9565df TargetServiceName=null EntryTimestamp=2022-04-26T06:26:15.945Z InvocationID=e52f6110-9c64-460c-ad46-c58c193e0d7f ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:26:16.250Z [qtp2085002312-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "895f054e-c898-492f-8100-718cf1b17db6" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "2.0" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=698914d1-40a9-4b00-80ae-57904e9565df TargetServiceName=null EntryTimestamp=2022-04-26T06:26:15.945Z InvocationID=e52f6110-9c64-460c-ad46-c58c193e0d7f ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:26:16.271Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "895f054e-c898-492f-8100-718cf1b17db6" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "2.1" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VduCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=698914d1-40a9-4b00-80ae-57904e9565df TargetServiceName=null EntryTimestamp=2022-04-26T06:26:15.945Z InvocationID=e52f6110-9c64-460c-ad46-c58c193e0d7f ErrorCode=500 lock resource with id d24dd8e0-d947-4e6f-85dc-9b1d14353b23 2022-04-26T06:26:16.273Z [qtp2085002312-14] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "895f054e-c898-492f-8100-718cf1b17db6" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "2.1" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VduCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=698914d1-40a9-4b00-80ae-57904e9565df TargetServiceName=null EntryTimestamp=2022-04-26T06:26:15.945Z InvocationID=e52f6110-9c64-460c-ad46-c58c193e0d7f ErrorCode=500 owner of resource d24dd8e0-d947-4e6f-85dc-9b1d14353b23 is jh0003 2022-04-26T06:26:16.809Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=b7bbf7ae-4445-44c0-a5ea-6a16b923a12c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:26:16.809Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=b7bbf7ae-4445-44c0-a5ea-6a16b923a12c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:26:16.939Z [qtp2085002312-14] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "698914d1-40a9-4b00-80ae-57904e9565df" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "2.1" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=698914d1-40a9-4b00-80ae-57904e9565df TargetServiceName=null EntryTimestamp=2022-04-26T06:26:15.945Z InvocationID=e52f6110-9c64-460c-ad46-c58c193e0d7f ErrorCode=500 owner of resource d24dd8e0-d947-4e6f-85dc-9b1d14353b23 is jh0003 2022-04-26T06:26:16.939Z [qtp2085002312-14] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "698914d1-40a9-4b00-80ae-57904e9565df" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "2.1" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=698914d1-40a9-4b00-80ae-57904e9565df TargetServiceName=null EntryTimestamp=2022-04-26T06:26:15.945Z InvocationID=e52f6110-9c64-460c-ad46-c58c193e0d7f ErrorCode=500 validate before certification change. resource name=VDU Cp, oldState=NOT_CERTIFIED_CHECKIN, owner userId=jh0003 2022-04-26T06:26:16.940Z [qtp2085002312-14] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "698914d1-40a9-4b00-80ae-57904e9565df" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "2.1" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=698914d1-40a9-4b00-80ae-57904e9565df TargetServiceName=null EntryTimestamp=2022-04-26T06:26:15.945Z InvocationID=e52f6110-9c64-460c-ad46-c58c193e0d7f ErrorCode=500 start performing certification change for resource d24dd8e0-d947-4e6f-85dc-9b1d14353b23 2022-04-26T06:26:17.045Z [qtp2085002312-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "698914d1-40a9-4b00-80ae-57904e9565df" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "" CURR_VERSION = "2.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "df70bdb7-2401-42e1-973a-22c27680e7a3.assettoscatemplate" CURR_ARTIFACT_UUID = "df70bdb7-2401-42e1-973a-22c27680e7a3.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCp-template.yml,0,df70bdb7-2401-42e1-973a-22c27680e7a3.assettoscatemplate,2,e576364e-d726-463d-af32-c339fb656b9c" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1ba20192-3585-411f-aee9-5dbf5321f865 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:15.945Z InvocationID=e52f6110-9c64-460c-ad46-c58c193e0d7f ErrorCode=500 Starting to create Csar for component VDU Cp 2022-04-26T06:26:17.242Z [qtp2085002312-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "698914d1-40a9-4b00-80ae-57904e9565df" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "" CURR_VERSION = "2.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "df70bdb7-2401-42e1-973a-22c27680e7a3.assettoscatemplate" CURR_ARTIFACT_UUID = "df70bdb7-2401-42e1-973a-22c27680e7a3.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCp-template.yml,0,df70bdb7-2401-42e1-973a-22c27680e7a3.assettoscatemplate,2,e576364e-d726-463d-af32-c339fb656b9c" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1ba20192-3585-411f-aee9-5dbf5321f865 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:15.945Z InvocationID=e52f6110-9c64-460c-ad46-c58c193e0d7f ErrorCode=500 Ended create Csar for component VDU Cp 2022-04-26T06:26:17.861Z [qtp2085002312-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "698914d1-40a9-4b00-80ae-57904e9565df" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "2.1" CURR_VERSION = "3.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bdf3f736-6563-45a9-81fa-2e4959215466 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:15.945Z InvocationID=e52f6110-9c64-460c-ad46-c58c193e0d7f ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:26:17.862Z [qtp2085002312-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "698914d1-40a9-4b00-80ae-57904e9565df" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "2.1" CURR_VERSION = "3.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bdf3f736-6563-45a9-81fa-2e4959215466 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:15.945Z InvocationID=e52f6110-9c64-460c-ad46-c58c193e0d7f ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:26:17.863Z [qtp2085002312-14] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "698914d1-40a9-4b00-80ae-57904e9565df" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "2.1" CURR_VERSION = "3.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bdf3f736-6563-45a9-81fa-2e4959215466 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:15.945Z InvocationID=e52f6110-9c64-460c-ad46-c58c193e0d7f ErrorCode=500 unlock component d24dd8e0-d947-4e6f-85dc-9b1d14353b23 2022-04-26T06:26:17.863Z [qtp2085002312-14] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "698914d1-40a9-4b00-80ae-57904e9565df" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "2.1" CURR_VERSION = "3.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bdf3f736-6563-45a9-81fa-2e4959215466 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:15.945Z InvocationID=e52f6110-9c64-460c-ad46-c58c193e0d7f ErrorCode=500 During change state, another component d24dd8e0-d947-4e6f-85dc-9b1d14353b23 has been created/updated 2022-04-26T06:26:17.876Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "698914d1-40a9-4b00-80ae-57904e9565df" INVARIANT_UUID = "cfa69c0b-19fe-4466-94ba-6483e62975d7" PREV_VERSION = "2.1" CURR_VERSION = "3.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1931 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bdf3f736-6563-45a9-81fa-2e4959215466 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:15.945Z InvocationID=e52f6110-9c64-460c-ad46-c58c193e0d7f ErrorCode=500 LogTimestamp=2022-04-26T06:26:17.876347Z ResponseDesc=Created Exiting. 2022-04-26T06:26:17.877Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:26:17.889Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=729ad957-44d4-4290-942d-e728f3db842e InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:17.889368Z InvocationID=191139c1-5fba-4412-97ed-58ed63f07186 LogTimestamp=2022-04-26T06:26:17.889445Z Entering 2022-04-26T06:26:17.890Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=729ad957-44d4-4290-942d-e728f3db842e InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:17.889368Z InvocationID=191139c1-5fba-4412-97ed-58ed63f07186 LogTimestamp=2022-04-26T06:26:17.889445Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2022-04-26T06:26:18.012Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:26:17.890Z InvocationID=f513af2f-abaa-4ded-aa96-08023066e17c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:26:18.013Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:26:17.890Z InvocationID=f513af2f-abaa-4ded-aa96-08023066e17c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:26:18.013Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:26:17.890Z InvocationID=f513af2f-abaa-4ded-aa96-08023066e17c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:26:18.013Z [qtp2085002312-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:26:17.890Z InvocationID=f513af2f-abaa-4ded-aa96-08023066e17c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:26:18.021Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2022-04-26T06:26:17.890Z InvocationID=f513af2f-abaa-4ded-aa96-08023066e17c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with name VirtualCp 2022-04-26T06:26:18.055Z [qtp2085002312-15] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Virtual Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3b458d51-8bc6-41ba-8d31-94d73ac69f17" INVARIANT_UUID = "24308a4e-78ae-4139-b113-a4c0093a1c88" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VirtualCp" RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3b458d51-8bc6-41ba-8d31-94d73ac69f17 EntryTimestamp=2022-04-26T06:26:17.890Z InvocationID=f513af2f-abaa-4ded-aa96-08023066e17c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:26:18.056Z [qtp2085002312-15] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Virtual Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3b458d51-8bc6-41ba-8d31-94d73ac69f17" INVARIANT_UUID = "24308a4e-78ae-4139-b113-a4c0093a1c88" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VirtualCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3b458d51-8bc6-41ba-8d31-94d73ac69f17 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:17.890Z InvocationID=f513af2f-abaa-4ded-aa96-08023066e17c ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:26:18.095Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Virtual Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3b458d51-8bc6-41ba-8d31-94d73ac69f17" INVARIANT_UUID = "24308a4e-78ae-4139-b113-a4c0093a1c88" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VirtualCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3b458d51-8bc6-41ba-8d31-94d73ac69f17 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:17.890Z InvocationID=f513af2f-abaa-4ded-aa96-08023066e17c ErrorCode=500 lock resource with id b697f7aa-c975-4067-8268-b39b73f26fae 2022-04-26T06:26:18.754Z [qtp2085002312-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Virtual Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3b458d51-8bc6-41ba-8d31-94d73ac69f17" INVARIANT_UUID = "24308a4e-78ae-4139-b113-a4c0093a1c88" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b697f7aa-c975-4067-8268-b39b73f26fae.assettoscatemplate" CURR_ARTIFACT_UUID = "b697f7aa-c975-4067-8268-b39b73f26fae.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VirtualCp-template.yml,0,b697f7aa-c975-4067-8268-b39b73f26fae.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=83a9cfbc-0141-4d0c-9938-59afd14fab2f TargetServiceName=null EntryTimestamp=2022-04-26T06:26:17.890Z InvocationID=f513af2f-abaa-4ded-aa96-08023066e17c ErrorCode=500 Starting to create Csar for component Virtual Cp 2022-04-26T06:26:18.883Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8576eb9b-808f-425f-85b2-5732cc8a5826 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:26:18.884Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8576eb9b-808f-425f-85b2-5732cc8a5826 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e0ae10 2022-04-26T06:26:18.884Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8576eb9b-808f-425f-85b2-5732cc8a5826 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:26:18.884Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8576eb9b-808f-425f-85b2-5732cc8a5826 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:26:18.958Z [qtp2085002312-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Virtual Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3b458d51-8bc6-41ba-8d31-94d73ac69f17" INVARIANT_UUID = "24308a4e-78ae-4139-b113-a4c0093a1c88" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b697f7aa-c975-4067-8268-b39b73f26fae.assettoscatemplate" CURR_ARTIFACT_UUID = "b697f7aa-c975-4067-8268-b39b73f26fae.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VirtualCp-template.yml,0,b697f7aa-c975-4067-8268-b39b73f26fae.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=83a9cfbc-0141-4d0c-9938-59afd14fab2f TargetServiceName=null EntryTimestamp=2022-04-26T06:26:17.890Z InvocationID=f513af2f-abaa-4ded-aa96-08023066e17c ErrorCode=500 Ended create Csar for component Virtual Cp 2022-04-26T06:26:18.959Z [qtp2085002312-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Virtual Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3b458d51-8bc6-41ba-8d31-94d73ac69f17" INVARIANT_UUID = "24308a4e-78ae-4139-b113-a4c0093a1c88" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b697f7aa-c975-4067-8268-b39b73f26fae.assettoscatemplate" CURR_ARTIFACT_UUID = "b697f7aa-c975-4067-8268-b39b73f26fae.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VirtualCp-template.yml,0,b697f7aa-c975-4067-8268-b39b73f26fae.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=83a9cfbc-0141-4d0c-9938-59afd14fab2f TargetServiceName=null EntryTimestamp=2022-04-26T06:26:17.890Z InvocationID=f513af2f-abaa-4ded-aa96-08023066e17c ErrorCode=500 lock resource with id b697f7aa-c975-4067-8268-b39b73f26fae 2022-04-26T06:26:19.570Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Virtual Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3b458d51-8bc6-41ba-8d31-94d73ac69f17" INVARIANT_UUID = "24308a4e-78ae-4139-b113-a4c0093a1c88" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b697f7aa-c975-4067-8268-b39b73f26fae.assettoscacsar" CURR_ARTIFACT_UUID = "b697f7aa-c975-4067-8268-b39b73f26fae.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VirtualCp-csar.csar,0,b697f7aa-c975-4067-8268-b39b73f26fae.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=1679 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9ea2c20b-d0d7-4194-88c7-f375954fb58a TargetServiceName=null EntryTimestamp=2022-04-26T06:26:17.890Z InvocationID=f513af2f-abaa-4ded-aa96-08023066e17c ErrorCode=500 LogTimestamp=2022-04-26T06:26:19.569672Z ResponseDesc=Created Exiting. 2022-04-26T06:26:19.570Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2022-04-26T06:26:19.641Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=76a33c06-87d5-42ff-b768-54cb5f5728e0 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/uploadType/grouptypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:19.641349Z InvocationID=b45fb81e-a22c-42a2-9ba7-a08b3c5a4c44 LogTimestamp=2022-04-26T06:26:19.641427Z Entering 2022-04-26T06:26:19.642Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=76a33c06-87d5-42ff-b768-54cb5f5728e0 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/uploadType/grouptypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:19.641349Z InvocationID=b45fb81e-a22c-42a2-9ba7-a08b3c5a4c44 LogTimestamp=2022-04-26T06:26:19.641427Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/uploadType/grouptypes HTTP/1.1 2022-04-26T06:26:19.669Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2022-04-26T06:26:19.642Z InvocationID=dbaf8bf5-c6b1-4f52-8f06-3c4ccf20a4dd PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send GROUP_TYPE : tosca.groups.root.1.0.grouptype to dao for create 2022-04-26T06:26:19.849Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2022-04-26T06:26:19.642Z InvocationID=dbaf8bf5-c6b1-4f52-8f06-3c4ccf20a4dd PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send GROUP_TYPE : org.openecomp.groups.heat.heatstack.1.0.grouptype to dao for create 2022-04-26T06:26:19.917Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=8576eb9b-808f-425f-85b2-5732cc8a5826 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:26:19.917Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=8576eb9b-808f-425f-85b2-5732cc8a5826 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:26:19.962Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2022-04-26T06:26:19.642Z InvocationID=dbaf8bf5-c6b1-4f52-8f06-3c4ccf20a4dd PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send GROUP_TYPE : org.openecomp.groups.vfmodule.1.0.grouptype to dao for create 2022-04-26T06:26:20.141Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2022-04-26T06:26:19.642Z InvocationID=dbaf8bf5-c6b1-4f52-8f06-3c4ccf20a4dd PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send GROUP_TYPE : org.openecomp.groups.networkcollection.1.0.grouptype to dao for create 2022-04-26T06:26:20.242Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2022-04-26T06:26:19.642Z InvocationID=dbaf8bf5-c6b1-4f52-8f06-3c4ccf20a4dd PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send GROUP_TYPE : org.openecomp.groups.vfcinstancegroup.1.0.grouptype to dao for create 2022-04-26T06:26:20.373Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:26:20.381Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:26:21.850Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2022-04-26T06:26:19.642Z InvocationID=dbaf8bf5-c6b1-4f52-8f06-3c4ccf20a4dd PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send GROUP_TYPE : tosca.groups.nfv.placementgroup.1.0.grouptype to dao for create 2022-04-26T06:26:22.741Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf055472-d95c-4896-8821-3d2a66bf7939 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:26:22.741Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf055472-d95c-4896-8821-3d2a66bf7939 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6aa05ec0 2022-04-26T06:26:22.741Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf055472-d95c-4896-8821-3d2a66bf7939 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:26:22.741Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf055472-d95c-4896-8821-3d2a66bf7939 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:26:23.347Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2022-04-26T06:26:19.642Z InvocationID=dbaf8bf5-c6b1-4f52-8f06-3c4ccf20a4dd PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send GROUP_TYPE : tosca.groups.nfv.nsplacementgroup.1.0.grouptype to dao for create 2022-04-26T06:26:23.757Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=bf055472-d95c-4896-8821-3d2a66bf7939 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:26:23.757Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=bf055472-d95c-4896-8821-3d2a66bf7939 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:26:24.243Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2022-04-26T06:26:19.642Z InvocationID=dbaf8bf5-c6b1-4f52-8f06-3c4ccf20a4dd PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send GROUP_TYPE : tosca.groups.nfv.vnffg.1.0.grouptype to dao for create 2022-04-26T06:26:24.341Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2022-04-26T06:26:19.642Z InvocationID=dbaf8bf5-c6b1-4f52-8f06-3c4ccf20a4dd PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb all GroupTypeDefinition(super=GroupTypeDataDefinition [uniqueId=tosca.groups.nfv.VNFFG.1.0.grouptype, type=tosca.groups.nfv.VNFFG, name=NFV VNF Forwarding Graph, icon=group-nfv-placementgroup, version=1.0, members=[tosca.nodes.nfv.NFP, tosca.nodes.nfv.VNF, tosca.nodes.nfv.NS, tosca.nodes.nfv.NsVirtualLink, tosca.nodes.nfv.NfpPositionElement], metadata=null, description=the VNFFG group type describes a topology of the NS or a portion of the NS, and optionally forwarding rules, applicable to the traffic conveyed over this topology, creationTime=1650954384250, modificationTime=1650954384250], properties=[PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=tosca.groups.nfv.VNFFG.1.0.grouptype.description, type=string, required=true, defaultValue=null, description=Human readable description of the group, schema=null, password=false, name=description, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, model=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=null, metadata=null), name=description, constraints=null]]], capabilities={}, model=null) were created successfully!!! 2022-04-26T06:26:25.643Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=43490317-b709-4013-8def-e8308b4f9636 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:26:25.643Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=43490317-b709-4013-8def-e8308b4f9636 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@484e578a 2022-04-26T06:26:25.643Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=43490317-b709-4013-8def-e8308b4f9636 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:26:25.643Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=43490317-b709-4013-8def-e8308b4f9636 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:26:26.665Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=43490317-b709-4013-8def-e8308b4f9636 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:26:26.665Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=43490317-b709-4013-8def-e8308b4f9636 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:26:27.971Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=8329 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2022-04-26T06:26:19.642Z InvocationID=dbaf8bf5-c6b1-4f52-8f06-3c4ccf20a4dd LogTimestamp=2022-04-26T06:26:27.971334Z ResponseDesc=Created Exiting. 2022-04-26T06:26:27.972Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/grouptypes HTTP/1.1 SC="201" 2022-04-26T06:26:27.983Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=77c97997-3e86-40e1-b395-582606287bcf InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/uploadType/policytypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:27.982834Z InvocationID=0c8d0133-325c-470e-938e-7a0fb4f3c446 LogTimestamp=2022-04-26T06:26:27.982893Z Entering 2022-04-26T06:26:27.983Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=77c97997-3e86-40e1-b395-582606287bcf InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/uploadType/policytypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:27.982834Z InvocationID=0c8d0133-325c-470e-938e-7a0fb4f3c446 LogTimestamp=2022-04-26T06:26:27.982893Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/uploadType/policytypes HTTP/1.1 2022-04-26T06:26:28.031Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : tosca.policies.root.1.0.policytype to dao for create 2022-04-26T06:26:28.057Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : tosca.policies.placement.1.0.policytype to dao for create 2022-04-26T06:26:28.241Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : tosca.policies.scaling.1.0.policytype to dao for create 2022-04-26T06:26:28.339Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : tosca.policies.update.1.0.policytype to dao for create 2022-04-26T06:26:28.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d770ddd1-7bce-4b4b-b3a1-126d63adf97f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:26:28.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d770ddd1-7bce-4b4b-b3a1-126d63adf97f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@63f61186 2022-04-26T06:26:28.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d770ddd1-7bce-4b4b-b3a1-126d63adf97f PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:26:28.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d770ddd1-7bce-4b4b-b3a1-126d63adf97f PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:26:28.449Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : tosca.policies.performance.1.0.policytype to dao for create 2022-04-26T06:26:28.640Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : org.openecomp.policies.placement.antilocate.1.0.policytype to dao for create 2022-04-26T06:26:28.849Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : org.openecomp.policies.placement.colocate.1.0.policytype to dao for create 2022-04-26T06:26:29.051Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : org.openecomp.policies.placement.valet.diversity.1.0.policytype to dao for create 2022-04-26T06:26:29.481Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=d770ddd1-7bce-4b4b-b3a1-126d63adf97f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:26:29.481Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=d770ddd1-7bce-4b4b-b3a1-126d63adf97f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:26:29.540Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : org.openecomp.policies.placement.valet.exclusivity.1.0.policytype to dao for create 2022-04-26T06:26:29.745Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : org.openecomp.policies.placement.valet.affinity.1.0.policytype to dao for create 2022-04-26T06:26:30.030Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : org.openecomp.policies.scaling.fixed.1.0.policytype to dao for create 2022-04-26T06:26:30.334Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : org.openecomp.policies.external.1.0.policytype to dao for create 2022-04-26T06:26:30.742Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : tosca.policies.nfv.instantiationlevels.1.0.policytype to dao for create 2022-04-26T06:26:31.341Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : tosca.policies.nfv.vduinstantiationlevels.1.0.policytype to dao for create 2022-04-26T06:26:31.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f70d231-7661-4fad-a3f6-f72a7619e630 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:26:31.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f70d231-7661-4fad-a3f6-f72a7619e630 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@74c832c2 2022-04-26T06:26:31.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f70d231-7661-4fad-a3f6-f72a7619e630 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:26:31.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f70d231-7661-4fad-a3f6-f72a7619e630 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:26:32.141Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : tosca.policies.nfv.virtuallinkinstantiationlevels.1.0.policytype to dao for create 2022-04-26T06:26:32.548Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : tosca.policies.nfv.scalingaspects.1.0.policytype to dao for create 2022-04-26T06:26:32.654Z [cluster2-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:26:32.655Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=6f70d231-7661-4fad-a3f6-f72a7619e630 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:26:32.656Z [cluster2-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 30000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:26:32.656Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=6f70d231-7661-4fad-a3f6-f72a7619e630 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:26:33.038Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : tosca.policies.nfv.vduscalingaspectdeltas.1.0.policytype to dao for create 2022-04-26T06:26:33.344Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : tosca.policies.nfv.virtuallinkbitratescalingaspectdeltas.1.0.policytype to dao for create 2022-04-26T06:26:33.840Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : tosca.policies.nfv.vduinitialdelta.1.0.policytype to dao for create 2022-04-26T06:26:34.331Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : tosca.policies.nfv.virtuallinkbitrateinitialdelta.1.0.policytype to dao for create 2022-04-26T06:26:34.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=264dfcf7-bcc2-48fa-9458-4b9aabfe8c42 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:26:34.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=264dfcf7-bcc2-48fa-9458-4b9aabfe8c42 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6c217b7c 2022-04-26T06:26:34.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=264dfcf7-bcc2-48fa-9458-4b9aabfe8c42 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:26:34.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=264dfcf7-bcc2-48fa-9458-4b9aabfe8c42 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:26:35.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:26:35.379Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:26:35.545Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : tosca.policies.nfv.securitygrouprule.1.0.policytype to dao for create 2022-04-26T06:26:35.661Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=264dfcf7-bcc2-48fa-9458-4b9aabfe8c42 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:26:35.662Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=264dfcf7-bcc2-48fa-9458-4b9aabfe8c42 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:26:36.044Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : tosca.policies.nfv.supportedvnfinterface.1.0.policytype to dao for create 2022-04-26T06:26:37.542Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08b9240b-e29e-4322-9997-861a76886cbf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:26:37.542Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08b9240b-e29e-4322-9997-861a76886cbf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1176cc22 2022-04-26T06:26:37.543Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08b9240b-e29e-4322-9997-861a76886cbf PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:26:37.543Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08b9240b-e29e-4322-9997-861a76886cbf PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:26:38.572Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=08b9240b-e29e-4322-9997-861a76886cbf ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:26:38.572Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=08b9240b-e29e-4322-9997-861a76886cbf ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:26:40.344Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78cc81af-f144-469f-b836-1ad3f00a2f18 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:26:40.344Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78cc81af-f144-469f-b836-1ad3f00a2f18 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@824e451 2022-04-26T06:26:40.345Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78cc81af-f144-469f-b836-1ad3f00a2f18 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:26:40.345Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78cc81af-f144-469f-b836-1ad3f00a2f18 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:26:40.761Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : tosca.policies.nfv.affinityrule.1.0.policytype to dao for create 2022-04-26T06:26:41.381Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : tosca.policies.nfv.antiaffinityrule.1.0.policytype to dao for create 2022-04-26T06:26:41.389Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=78cc81af-f144-469f-b836-1ad3f00a2f18 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:26:41.390Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=78cc81af-f144-469f-b836-1ad3f00a2f18 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:26:42.040Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : tosca.policies.nfv.nsaffinityrule.1.0.policytype to dao for create 2022-04-26T06:26:42.839Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : tosca.policies.nfv.nsantiaffinityrule.1.0.policytype to dao for create 2022-04-26T06:26:42.872Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : tosca.policies.nfv.abstract.securitygrouprule.1.0.policytype to dao for create 2022-04-26T06:26:42.937Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6842f97-0d5b-44b4-97a6-9e09530bcf96 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:26:42.937Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6842f97-0d5b-44b4-97a6-9e09530bcf96 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@66e235fc 2022-04-26T06:26:42.937Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6842f97-0d5b-44b4-97a6-9e09530bcf96 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:26:42.937Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6842f97-0d5b-44b4-97a6-9e09530bcf96 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:26:43.011Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : tosca.policies.nfv.nssecuritygrouprule.1.0.policytype to dao for create 2022-04-26T06:26:43.073Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : tosca.policies.nfv.nfprule.1.0.policytype to dao for create 2022-04-26T06:26:43.135Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : tosca.policies.nfv.nsmonitoring.1.0.policytype to dao for create 2022-04-26T06:26:43.165Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : tosca.policies.nfv.vnfmonitoring.1.0.policytype to dao for create 2022-04-26T06:26:43.185Z [qtp2085002312-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb all PolicyTypeDefinition(super=PolicyTypeDataDefinition [uniqueId=tosca.policies.nfv.VnfMonitoring.1.0.policytype, type=tosca.policies.nfv.VnfMonitoring, name=NFV VnfMonitoring, icon=policy-nfv-anti-affinityrule, version=1.0, targets=[tosca.nodes.nfv.VNF], metadata=null, description=Policy type is used to identify information to be monitored during the lifetime of a VNF instance as defined in ETSI GS NFV-IFA 014 [2]., creationTime=1650954403166, modificationTime=1650954403166], properties=[PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=tosca.policies.nfv.VnfMonitoring.1.0.policytype.vnf_monitoring_parameters, type=list, required=true, defaultValue=null, description=Specifies a virtualised resource related performance metric to be monitored on the NS level., schema=SchemaDefinition [derivedFrom='null, constraints=null, properties=null, property=PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=null, type=tosca.datatypes.nfv.VnfMonitoringParameter, required=false, defaultValue=null, description=null, schema=null, password=false, name=null, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, model=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=null, metadata=null), name=null, constraints=null]]], password=false, name=vnf_monitoring_parameters, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, model=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=[{"minLength":1}], metadata=null), name=vnf_monitoring_parameters, constraints=[org.openecomp.sdc.be.model.tosca.constraints.MinLengthConstraint@58398183]]]], model=null) were created successfully!!! 2022-04-26T06:26:43.650Z [cluster1-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:26:43.651Z [cluster1-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 64000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:26:43.945Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f6842f97-0d5b-44b4-97a6-9e09530bcf96 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:26:43.945Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f6842f97-0d5b-44b4-97a6-9e09530bcf96 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:26:43.986Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=16003 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:26:27.983Z InvocationID=feb6c07a-ca05-4007-8f9d-cb90f3e84d98 LogTimestamp=2022-04-26T06:26:43.986484Z ResponseDesc=Created Exiting. 2022-04-26T06:26:43.987Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/policytypes HTTP/1.1 SC="201" 2022-04-26T06:26:44.004Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=3cd2c7c1-7fa3-4ce7-beab-6ab7b880e825 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=2 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/model ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:44.002130Z InvocationID=6a40ea84-dc94-453f-a7f2-858eeb17b580 LogTimestamp=2022-04-26T06:26:44.004564Z Entering 2022-04-26T06:26:44.005Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=3cd2c7c1-7fa3-4ce7-beab-6ab7b880e825 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=2 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/model ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:44.002130Z InvocationID=6a40ea84-dc94-453f-a7f2-858eeb17b580 LogTimestamp=2022-04-26T06:26:44.004564Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 GET /sdc2/rest/v1/catalog/model HTTP/1.1 2022-04-26T06:26:44.089Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=84 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/model EntryTimestamp=2022-04-26T06:26:44.005Z InvocationID=a634e812-b09b-4ab9-be48-8683628106c0 LogTimestamp=2022-04-26T06:26:44.089533Z ResponseDesc=OK Exiting. 2022-04-26T06:26:44.090Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/catalog/model HTTP/1.1 SC="200" 2022-04-26T06:26:44.100Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=e46f414e-7208-439e-85ae-e078ad0da859 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/model ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:44.100292Z InvocationID=e7b0ed81-88fc-4007-baed-ecfdb6aada68 LogTimestamp=2022-04-26T06:26:44.100374Z Entering 2022-04-26T06:26:44.101Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=e46f414e-7208-439e-85ae-e078ad0da859 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/model ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:44.100292Z InvocationID=e7b0ed81-88fc-4007-baed-ecfdb6aada68 LogTimestamp=2022-04-26T06:26:44.100374Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/model HTTP/1.1 2022-04-26T06:26:44.181Z [qtp2085002312-14] INFO com.jcabi.aspects.aj.MethodValidator RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/model EntryTimestamp=2022-04-26T06:26:44.101Z InvocationID=e7f21c9e-1078-4aae-8627-84756bc388c3 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb JSR-303 validator org.hibernate.validator.internal.engine.ValidatorImpl instantiated by jcabi-aspects 0.20.1/f0aff93 2022-04-26T06:26:44.483Z [qtp2085002312-14] INFO org.reflections.Reflections RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/model EntryTimestamp=2022-04-26T06:26:44.101Z InvocationID=e7f21c9e-1078-4aae-8627-84756bc388c3 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Reflections took 175 ms to scan 315 urls, producing 0 keys and 0 values 2022-04-26T06:26:45.230Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1129 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/model EntryTimestamp=2022-04-26T06:26:44.101Z InvocationID=e7f21c9e-1078-4aae-8627-84756bc388c3 LogTimestamp=2022-04-26T06:26:45.230021Z ResponseDesc=Created Exiting. 2022-04-26T06:26:45.230Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/model HTTP/1.1 SC="201" 2022-04-26T06:26:45.251Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ef413522-da1f-433d-b246-40a6e944a991 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:45.250589Z InvocationID=d729d82a-2b80-436a-b80e-a96c67bc0186 LogTimestamp=2022-04-26T06:26:45.250652Z Entering 2022-04-26T06:26:45.251Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ef413522-da1f-433d-b246-40a6e944a991 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:45.250589Z InvocationID=d729d82a-2b80-436a-b80e-a96c67bc0186 LogTimestamp=2022-04-26T06:26:45.250652Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2022-04-26T06:26:45.291Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.tosca.datatypes.Root.datatype to dao for create 2022-04-26T06:26:45.293Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.306Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.integer.datatype to dao for create 2022-04-26T06:26:45.311Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.315Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.string.datatype to dao for create 2022-04-26T06:26:45.319Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.321Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.boolean.datatype to dao for create 2022-04-26T06:26:45.324Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.327Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.float.datatype to dao for create 2022-04-26T06:26:45.330Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.332Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.list.datatype to dao for create 2022-04-26T06:26:45.334Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.337Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.map.datatype to dao for create 2022-04-26T06:26:45.340Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.342Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.json.datatype to dao for create 2022-04-26T06:26:45.347Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.349Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.scalar-unit.datatype to dao for create 2022-04-26T06:26:45.352Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.359Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.scalar-unit.size.datatype to dao for create 2022-04-26T06:26:45.362Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.365Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.scalar-unit.time.datatype to dao for create 2022-04-26T06:26:45.370Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.374Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.scalar-unit.frequency.datatype to dao for create 2022-04-26T06:26:45.377Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.380Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.onap.datatypes.ToscaConceptIdentifier.datatype to dao for create 2022-04-26T06:26:45.399Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.402Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.onap.datatype.acm.Target.datatype to dao for create 2022-04-26T06:26:45.409Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.411Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.onap.datatype.acm.Actor.datatype to dao for create 2022-04-26T06:26:45.427Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.429Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.onap.datatype.acm.Operation.datatype to dao for create 2022-04-26T06:26:45.457Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.460Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.onap.datatypes.monitoring.managedObjectDNsBasic.datatype to dao for create 2022-04-26T06:26:45.466Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.468Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.onap.datatypes.monitoring.managedObjectDNsBasics.datatype to dao for create 2022-04-26T06:26:45.474Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.478Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.onap.datatypes.monitoring.measurementGroup.datatype to dao for create 2022-04-26T06:26:45.484Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.487Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.onap.datatypes.monitoring.measurementGroups.datatype to dao for create 2022-04-26T06:26:45.492Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.495Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.onap.datatypes.monitoring.measurementType.datatype to dao for create 2022-04-26T06:26:45.499Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.502Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.onap.datatypes.monitoring.measurementTypes.datatype to dao for create 2022-04-26T06:26:45.506Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.508Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.onap.datatypes.monitoring.nfFilter.datatype to dao for create 2022-04-26T06:26:45.520Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.522Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.onap.datatypes.monitoring.subscription.datatype to dao for create 2022-04-26T06:26:45.538Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.541Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.org.onap.datatypes.policy.clamp.acm.httpAutomationCompositionElement.RestRequest.datatype to dao for create 2022-04-26T06:26:45.558Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.560Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : AUTOMATION COMPOSITION.org.onap.datatypes.policy.clamp.acm.httpAutomationCompositionElement.ConfigurationEntity.datatype to dao for create 2022-04-26T06:26:45.568Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:45.570Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb all DataTypeDefinition(derivedFrom=null, constraints=null, properties=[PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=AUTOMATION COMPOSITION.org.onap.datatypes.policy.clamp.acm.httpAutomationCompositionElement.ConfigurationEntity.datatype.restSequence, type=list, required=false, defaultValue=null, description=A sequence of REST commands to send to the REST endpoint, schema=SchemaDefinition [derivedFrom='null, constraints=null, properties=null, property=PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=null, type=org.onap.datatypes.policy.clamp.acm.httpAutomationCompositionElement.RestRequest, required=false, defaultValue=null, description=null, schema=null, password=false, name=null, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, model=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=null, metadata=null), name=null, constraints=null]]], password=false, name=restSequence, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, model=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=null, metadata=null), name=restSequence, constraints=null]], PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=AUTOMATION COMPOSITION.org.onap.datatypes.policy.clamp.acm.httpAutomationCompositionElement.ConfigurationEntity.datatype.configurationEntityId, type=onap.datatypes.ToscaConceptIdentifier, required=true, defaultValue=null, description=The name and version of a Configuration Entity to be handled by the HTTP Automation Composition Element, schema=null, password=false, name=configurationEntityId, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, model=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=null, metadata=null), name=configurationEntityId, constraints=null]]]) were created successfully!!! 2022-04-26T06:26:45.792Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50c0d5df-708b-4866-8ee0-a4bea28e4fff PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:26:45.792Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50c0d5df-708b-4866-8ee0-a4bea28e4fff PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6a93306b 2022-04-26T06:26:45.792Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50c0d5df-708b-4866-8ee0-a4bea28e4fff PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:26:45.792Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50c0d5df-708b-4866-8ee0-a4bea28e4fff PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:26:46.829Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=50c0d5df-708b-4866-8ee0-a4bea28e4fff ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:26:46.829Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=50c0d5df-708b-4866-8ee0-a4bea28e4fff ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:26:47.474Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2222 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:26:45.251Z InvocationID=745b69be-88ed-48e8-be35-4b237626c8f8 LogTimestamp=2022-04-26T06:26:47.473751Z ResponseDesc=Created Exiting. 2022-04-26T06:26:47.474Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="201" 2022-04-26T06:26:47.485Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=de602e66-9b55-487e-bf49-3e102f6bd4df InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/uploadType/capability ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:47.484909Z InvocationID=ee6984b2-5179-4d87-82e0-72b07366c5ae LogTimestamp=2022-04-26T06:26:47.484951Z Entering 2022-04-26T06:26:47.485Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=de602e66-9b55-487e-bf49-3e102f6bd4df InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/uploadType/capability ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:47.484909Z InvocationID=ee6984b2-5179-4d87-82e0-72b07366c5ae LogTimestamp=2022-04-26T06:26:47.484951Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/uploadType/capability HTTP/1.1 2022-04-26T06:26:47.536Z [qtp2085002312-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:26:47.485Z InvocationID=5f899e31-8b76-4b14-95dd-8deb07f9eec0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : AUTOMATION COMPOSITION.tosca.capabilities.Root to dao for create 2022-04-26T06:26:47.539Z [qtp2085002312-18] INFO o.o.s.b.m.o.i.CapabilityTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:26:47.485Z InvocationID=5f899e31-8b76-4b14-95dd-8deb07f9eec0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=CapabilityType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:47.549Z [qtp2085002312-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:26:47.485Z InvocationID=5f899e31-8b76-4b14-95dd-8deb07f9eec0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : AUTOMATION COMPOSITION.tosca.capabilities.Node to dao for create 2022-04-26T06:26:47.550Z [qtp2085002312-18] INFO o.o.s.b.m.o.i.CapabilityTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:26:47.485Z InvocationID=5f899e31-8b76-4b14-95dd-8deb07f9eec0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=CapabilityType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:47.554Z [qtp2085002312-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:26:47.485Z InvocationID=5f899e31-8b76-4b14-95dd-8deb07f9eec0 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb all CapabilityTypeDefinition(super=CapabilityTypeDataDefinition [uniqueId=AUTOMATION COMPOSITION.tosca.capabilities.Node, description=null, type=tosca.capabilities.Node, validSourceTypes=null, version=null, creationTime=1650954407550, modificationTime=1650954407550], derivedFrom=tosca.capabilities.Root, properties=null, model=AUTOMATION COMPOSITION) were created successfully!!! 2022-04-26T06:26:48.078Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=593 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:26:47.485Z InvocationID=5f899e31-8b76-4b14-95dd-8deb07f9eec0 LogTimestamp=2022-04-26T06:26:48.078014Z ResponseDesc=Created Exiting. 2022-04-26T06:26:48.078Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/capability HTTP/1.1 SC="201" 2022-04-26T06:26:48.085Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c8e500ab-6fcd-4b8e-bb04-610b34280e96 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/uploadType/relationship ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:48.085Z InvocationID=f54c6e80-91f0-494b-9fa1-87daacc899ff LogTimestamp=2022-04-26T06:26:48.085031Z Entering 2022-04-26T06:26:48.085Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c8e500ab-6fcd-4b8e-bb04-610b34280e96 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/uploadType/relationship ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:48.085Z InvocationID=f54c6e80-91f0-494b-9fa1-87daacc899ff LogTimestamp=2022-04-26T06:26:48.085031Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/uploadType/relationship HTTP/1.1 2022-04-26T06:26:48.096Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:26:48.085Z InvocationID=d9ffe8dc-2fa6-437e-8851-f44780fe87f3 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : AUTOMATION COMPOSITION.tosca.relationships.Root to dao for create 2022-04-26T06:26:48.099Z [qtp2085002312-13] INFO o.o.s.b.m.o.i.RelationshipTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:26:48.085Z InvocationID=d9ffe8dc-2fa6-437e-8851-f44780fe87f3 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=RelationshipType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:48.615Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:26:48.085Z InvocationID=d9ffe8dc-2fa6-437e-8851-f44780fe87f3 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : AUTOMATION COMPOSITION.tosca.relationships.DependsOn to dao for create 2022-04-26T06:26:48.617Z [qtp2085002312-13] INFO o.o.s.b.m.o.i.RelationshipTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:26:48.085Z InvocationID=d9ffe8dc-2fa6-437e-8851-f44780fe87f3 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=RelationshipType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:48.880Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=817e19aa-3649-4efe-a378-ebfb1d8fd7f8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:26:48.880Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=817e19aa-3649-4efe-a378-ebfb1d8fd7f8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4b9c678a 2022-04-26T06:26:48.880Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=817e19aa-3649-4efe-a378-ebfb1d8fd7f8 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:26:48.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=817e19aa-3649-4efe-a378-ebfb1d8fd7f8 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:26:49.154Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:26:48.085Z InvocationID=d9ffe8dc-2fa6-437e-8851-f44780fe87f3 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb all RelationshipTypeDefinition(super=RelationshipTypeDataDefinition [uniqueId=AUTOMATION COMPOSITION.tosca.relationships.DependsOn, description=This type represents a general dependency relationship between two nodes., type=tosca.relationships.DependsOn, validSourceTypes=null, version=null, originUI=null, creationTime=1650954408616, modificationTime=1650954408616], derivedFrom=tosca.relationships.Root, properties=null, model=AUTOMATION COMPOSITION) were created successfully!!! 2022-04-26T06:26:49.156Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1071 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:26:48.085Z InvocationID=d9ffe8dc-2fa6-437e-8851-f44780fe87f3 LogTimestamp=2022-04-26T06:26:49.156097Z ResponseDesc=Created Exiting. 2022-04-26T06:26:49.156Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/relationship HTTP/1.1 SC="201" 2022-04-26T06:26:49.164Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f51acca9-803a-475e-b178-3a2140dfb23f InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/uploadType/interfaceLifecycle ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:49.163908Z InvocationID=ace2b891-0ebc-40c8-b850-8e922f198416 LogTimestamp=2022-04-26T06:26:49.163962Z Entering 2022-04-26T06:26:49.164Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f51acca9-803a-475e-b178-3a2140dfb23f InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/uploadType/interfaceLifecycle ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:49.163908Z InvocationID=ace2b891-0ebc-40c8-b850-8e922f198416 LogTimestamp=2022-04-26T06:26:49.163962Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/uploadType/interfaceLifecycle HTTP/1.1 2022-04-26T06:26:49.175Z [qtp2085002312-15] INFO o.o.s.b.c.i.InterfaceLifecycleTypeImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2022-04-26T06:26:49.164Z InvocationID=54b2e95f-60c2-40f6-af7b-ed545195cefc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send interfaceDefinition tosca.interfaces.Root to dao for create 2022-04-26T06:26:49.179Z [qtp2085002312-15] INFO o.o.s.b.m.o.i.InterfaceLifecycleOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2022-04-26T06:26:49.164Z InvocationID=54b2e95f-60c2-40f6-af7b-ed545195cefc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=Interface, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:49.748Z [qtp2085002312-15] INFO o.o.s.b.c.i.InterfaceLifecycleTypeImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2022-04-26T06:26:49.164Z InvocationID=54b2e95f-60c2-40f6-af7b-ed545195cefc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send interfaceDefinition tosca.interfaces.node.lifecycle.Standard to dao for create 2022-04-26T06:26:49.762Z [qtp2085002312-15] INFO o.o.s.b.m.o.i.InterfaceLifecycleOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2022-04-26T06:26:49.164Z InvocationID=54b2e95f-60c2-40f6-af7b-ed545195cefc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=Interface, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:49.900Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=817e19aa-3649-4efe-a378-ebfb1d8fd7f8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:26:49.901Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=817e19aa-3649-4efe-a378-ebfb1d8fd7f8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:26:50.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:26:50.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:26:50.638Z [qtp2085002312-15] INFO o.o.s.b.c.i.InterfaceLifecycleTypeImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2022-04-26T06:26:49.164Z InvocationID=54b2e95f-60c2-40f6-af7b-ed545195cefc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send interfaceDefinition tosca.interfaces.relationship.Configure to dao for create 2022-04-26T06:26:50.744Z [qtp2085002312-15] INFO o.o.s.b.m.o.i.InterfaceLifecycleOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2022-04-26T06:26:49.164Z InvocationID=54b2e95f-60c2-40f6-af7b-ed545195cefc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=Interface, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:26:52.036Z [qtp2085002312-15] INFO o.o.s.b.c.i.InterfaceLifecycleTypeImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2022-04-26T06:26:49.164Z InvocationID=54b2e95f-60c2-40f6-af7b-ed545195cefc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lifecycle types were created successfully!!! 2022-04-26T06:26:52.039Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2875 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2022-04-26T06:26:49.164Z InvocationID=54b2e95f-60c2-40f6-af7b-ed545195cefc LogTimestamp=2022-04-26T06:26:52.039503Z ResponseDesc=Created Exiting. 2022-04-26T06:26:52.040Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/interfaceLifecycle HTTP/1.1 SC="201" 2022-04-26T06:26:52.047Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a985bdc0-a973-42d7-8f0f-91e91e245d48 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/resource/import ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:52.046866Z InvocationID=d7f825c3-aa98-4f0c-b8f7-d94c255028ca LogTimestamp=2022-04-26T06:26:52.046905Z Entering 2022-04-26T06:26:52.047Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a985bdc0-a973-42d7-8f0f-91e91e245d48 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/upload/resource/import ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:26:52.046866Z InvocationID=d7f825c3-aa98-4f0c-b8f7-d94c255028ca LogTimestamp=2022-04-26T06:26:52.046905Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/upload/resource/import HTTP/1.1 2022-04-26T06:26:52.539Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb5e0ca2-597f-40a7-bde3-758f569df41d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:26:52.539Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb5e0ca2-597f-40a7-bde3-758f569df41d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2fc57ac5 2022-04-26T06:26:52.539Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb5e0ca2-597f-40a7-bde3-758f569df41d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:26:52.539Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb5e0ca2-597f-40a7-bde3-758f569df41d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:26:52.643Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:26:52.643Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:26:52.643Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:26:52.643Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:26:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:26:52.686Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:26:52.686Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:26:52.847Z [qtp2085002312-20] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Create Resource - validateLifecycleTypesCreate ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import TargetServiceName=null EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd ErrorCode=400 Interface tosca.interfaces.node.lifecycle.standard required is missing 2022-04-26T06:26:52.848Z [qtp2085002312-20] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Create Resource - validateLifecycleTypesCreate ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import TargetServiceName=null EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd ErrorCode=500 Operation towards database failed. 2022-04-26T06:26:52.848Z [qtp2085002312-20] WARN o.o.s.e.AbstractSdncException RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import TargetServiceName=null EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd ErrorCode=900 2022-04-26T06:26:53.576Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=fb5e0ca2-597f-40a7-bde3-758f569df41d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:26:53.577Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=fb5e0ca2-597f-40a7-bde3-758f569df41d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:26:55.837Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a2cc121a-4b17-47d2-82f3-ee5569422d41 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:26:55.837Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a2cc121a-4b17-47d2-82f3-ee5569422d41 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@ca1cbe2 2022-04-26T06:26:55.837Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a2cc121a-4b17-47d2-82f3-ee5569422d41 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:26:55.837Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a2cc121a-4b17-47d2-82f3-ee5569422d41 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:26:56.352Z [JanusGraph Cluster-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:26:56.353Z [JanusGraph Cluster-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 64000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:26:56.878Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=a2cc121a-4b17-47d2-82f3-ee5569422d41 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:26:56.878Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=a2cc121a-4b17-47d2-82f3-ee5569422d41 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:26:58.454Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a645049-63d6-44b9-bdf4-55af603f4992 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:26:58.454Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a645049-63d6-44b9-bdf4-55af603f4992 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6436f2ae 2022-04-26T06:26:58.454Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a645049-63d6-44b9-bdf4-55af603f4992 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:26:58.454Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a645049-63d6-44b9-bdf4-55af603f4992 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:26:59.500Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=6a645049-63d6-44b9-bdf4-55af603f4992 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:26:59.501Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=6a645049-63d6-44b9-bdf4-55af603f4992 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:27:01.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec80c486-5536-46bb-aef8-2dc994a453ce PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:27:01.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec80c486-5536-46bb-aef8-2dc994a453ce PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8cb3d05 2022-04-26T06:27:01.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec80c486-5536-46bb-aef8-2dc994a453ce PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:27:01.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec80c486-5536-46bb-aef8-2dc994a453ce PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:27:02.665Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ec80c486-5536-46bb-aef8-2dc994a453ce ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:27:02.665Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ec80c486-5536-46bb-aef8-2dc994a453ce ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:27:04.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e6a4073-8da0-41f7-ac65-f5539b38e33c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:27:04.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e6a4073-8da0-41f7-ac65-f5539b38e33c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@368e1e65 2022-04-26T06:27:04.531Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e6a4073-8da0-41f7-ac65-f5539b38e33c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:27:04.531Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e6a4073-8da0-41f7-ac65-f5539b38e33c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:27:05.368Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:27:05.377Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:27:05.549Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=2e6a4073-8da0-41f7-ac65-f5539b38e33c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:27:05.550Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=2e6a4073-8da0-41f7-ac65-f5539b38e33c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:27:07.635Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09ee7863-51a0-485a-a385-994df3ba898d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:27:07.636Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09ee7863-51a0-485a-a385-994df3ba898d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@55328b1f 2022-04-26T06:27:07.636Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09ee7863-51a0-485a-a385-994df3ba898d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:27:07.636Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09ee7863-51a0-485a-a385-994df3ba898d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:27:07.642Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:27:08.649Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=09ee7863-51a0-485a-a385-994df3ba898d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:27:08.649Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=09ee7863-51a0-485a-a385-994df3ba898d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:27:09.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc5457ad-b5f6-4ba4-872a-4b0e10f40418 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:27:09.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc5457ad-b5f6-4ba4-872a-4b0e10f40418 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7fcb0b7 2022-04-26T06:27:09.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc5457ad-b5f6-4ba4-872a-4b0e10f40418 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:27:09.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc5457ad-b5f6-4ba4-872a-4b0e10f40418 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:27:10.358Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Root" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "0e8fd4d7-a592-4897-872a-477513d4c2d7" INVARIANT_UUID = "b14d36ea-2ebb-4e8d-8641-01b87b06e64c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a48d7da6-1975-4258-8423-6c10ff695f77.assettoscatemplate" CURR_ARTIFACT_UUID = "a48d7da6-1975-4258-8423-6c10ff695f77.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Root-template.yml,0,a48d7da6-1975-4258-8423-6c10ff695f77.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=f8f731a6-1272-48d7-9506-3cf0aa4db76d TargetServiceName=null EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd ErrorCode=900 LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:27:10.358Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Root" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "0e8fd4d7-a592-4897-872a-477513d4c2d7" INVARIANT_UUID = "b14d36ea-2ebb-4e8d-8641-01b87b06e64c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a48d7da6-1975-4258-8423-6c10ff695f77.assettoscatemplate" CURR_ARTIFACT_UUID = "a48d7da6-1975-4258-8423-6c10ff695f77.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Root-template.yml,0,a48d7da6-1975-4258-8423-6c10ff695f77.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=f8f731a6-1272-48d7-9506-3cf0aa4db76d TargetServiceName=null EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd ErrorCode=900 Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:27:10.359Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Root" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "0e8fd4d7-a592-4897-872a-477513d4c2d7" INVARIANT_UUID = "b14d36ea-2ebb-4e8d-8641-01b87b06e64c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a48d7da6-1975-4258-8423-6c10ff695f77.assettoscatemplate" CURR_ARTIFACT_UUID = "a48d7da6-1975-4258-8423-6c10ff695f77.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Root-template.yml,0,a48d7da6-1975-4258-8423-6c10ff695f77.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=f8f731a6-1272-48d7-9506-3cf0aa4db76d TargetServiceName=null EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd ErrorCode=900 Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:27:10.359Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Root" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "0e8fd4d7-a592-4897-872a-477513d4c2d7" INVARIANT_UUID = "b14d36ea-2ebb-4e8d-8641-01b87b06e64c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a48d7da6-1975-4258-8423-6c10ff695f77.assettoscatemplate" CURR_ARTIFACT_UUID = "a48d7da6-1975-4258-8423-6c10ff695f77.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Root-template.yml,0,a48d7da6-1975-4258-8423-6c10ff695f77.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=f8f731a6-1272-48d7-9506-3cf0aa4db76d TargetServiceName=null EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd ErrorCode=900 Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:27:10.384Z [qtp2085002312-20] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Participant" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "45881b99-5799-4879-9441-833b7ab2b154" INVARIANT_UUID = "14283c1c-fc69-4edc-901c-a2d8ff06f855" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.onap.policy.clamp.acm.Participant" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=45881b99-5799-4879-9441-833b7ab2b154 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd ErrorCode=900 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:27:10.385Z [qtp2085002312-20] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Participant" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "45881b99-5799-4879-9441-833b7ab2b154" INVARIANT_UUID = "14283c1c-fc69-4edc-901c-a2d8ff06f855" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.onap.policy.clamp.acm.Participant" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=45881b99-5799-4879-9441-833b7ab2b154 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:27:10.409Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Participant" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "45881b99-5799-4879-9441-833b7ab2b154" INVARIANT_UUID = "14283c1c-fc69-4edc-901c-a2d8ff06f855" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "99a025e2-0051-4324-b297-02651b0b2b0d.assettoscatemplate" CURR_ARTIFACT_UUID = "99a025e2-0051-4324-b297-02651b0b2b0d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Participant-template.yml,0,99a025e2-0051-4324-b297-02651b0b2b0d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=cd34b700-c20b-4874-8b5b-75e1551a7188 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd ErrorCode=500 Starting to create Csar for component Participant 2022-04-26T06:27:10.426Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Participant" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "45881b99-5799-4879-9441-833b7ab2b154" INVARIANT_UUID = "14283c1c-fc69-4edc-901c-a2d8ff06f855" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "99a025e2-0051-4324-b297-02651b0b2b0d.assettoscatemplate" CURR_ARTIFACT_UUID = "99a025e2-0051-4324-b297-02651b0b2b0d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Participant-template.yml,0,99a025e2-0051-4324-b297-02651b0b2b0d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=cd34b700-c20b-4874-8b5b-75e1551a7188 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd ErrorCode=500 Ended create Csar for component Participant 2022-04-26T06:27:10.457Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Participant" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "45881b99-5799-4879-9441-833b7ab2b154" INVARIANT_UUID = "14283c1c-fc69-4edc-901c-a2d8ff06f855" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "99a025e2-0051-4324-b297-02651b0b2b0d.assettoscacsar" CURR_ARTIFACT_UUID = "99a025e2-0051-4324-b297-02651b0b2b0d.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Participant-csar.csar,0,99a025e2-0051-4324-b297-02651b0b2b0d.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=5a1cde80-cd3b-4024-9b2f-fd5a352a9f42 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd ErrorCode=500 LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:27:10.457Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Participant" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "45881b99-5799-4879-9441-833b7ab2b154" INVARIANT_UUID = "14283c1c-fc69-4edc-901c-a2d8ff06f855" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "99a025e2-0051-4324-b297-02651b0b2b0d.assettoscacsar" CURR_ARTIFACT_UUID = "99a025e2-0051-4324-b297-02651b0b2b0d.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Participant-csar.csar,0,99a025e2-0051-4324-b297-02651b0b2b0d.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=5a1cde80-cd3b-4024-9b2f-fd5a352a9f42 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd ErrorCode=500 Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:27:10.457Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Participant" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "45881b99-5799-4879-9441-833b7ab2b154" INVARIANT_UUID = "14283c1c-fc69-4edc-901c-a2d8ff06f855" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "99a025e2-0051-4324-b297-02651b0b2b0d.assettoscacsar" CURR_ARTIFACT_UUID = "99a025e2-0051-4324-b297-02651b0b2b0d.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Participant-csar.csar,0,99a025e2-0051-4324-b297-02651b0b2b0d.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=5a1cde80-cd3b-4024-9b2f-fd5a352a9f42 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd ErrorCode=500 Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:27:10.457Z [qtp2085002312-20] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Participant" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "45881b99-5799-4879-9441-833b7ab2b154" INVARIANT_UUID = "14283c1c-fc69-4edc-901c-a2d8ff06f855" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "99a025e2-0051-4324-b297-02651b0b2b0d.assettoscacsar" CURR_ARTIFACT_UUID = "99a025e2-0051-4324-b297-02651b0b2b0d.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Participant-csar.csar,0,99a025e2-0051-4324-b297-02651b0b2b0d.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=5a1cde80-cd3b-4024-9b2f-fd5a352a9f42 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd ErrorCode=500 Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:27:10.489Z [qtp2085002312-20] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "AutomationCompositionElement" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2be9d5b6-cd77-411e-b841-3ee480311452" INVARIANT_UUID = "622f3043-48cc-4a54-a931-4b338101b3a9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.onap.policy.clamp.acm.AutomationCompositionElement" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=2be9d5b6-cd77-411e-b841-3ee480311452 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:27:10.490Z [qtp2085002312-20] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "AutomationCompositionElement" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2be9d5b6-cd77-411e-b841-3ee480311452" INVARIANT_UUID = "622f3043-48cc-4a54-a931-4b338101b3a9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.onap.policy.clamp.acm.AutomationCompositionElement" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=2be9d5b6-cd77-411e-b841-3ee480311452 TargetServiceName=null EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:27:10.509Z [qtp2085002312-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "AutomationCompositionElement" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2be9d5b6-cd77-411e-b841-3ee480311452" INVARIANT_UUID = "622f3043-48cc-4a54-a931-4b338101b3a9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2e88610e-aba3-4d43-84f2-bba006643ae0.assettoscatemplate" CURR_ARTIFACT_UUID = "2e88610e-aba3-4d43-84f2-bba006643ae0.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Automationcompositionelement-template.yml,0,2e88610e-aba3-4d43-84f2-bba006643ae0.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=c192d11e-1599-4fe9-9c17-4c8eee5d6bab TargetServiceName=null EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd ErrorCode=500 Starting to create Csar for component AutomationCompositionElement 2022-04-26T06:27:10.539Z [qtp2085002312-20] WARN o.o.s.e.AbstractSdncException AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "AutomationCompositionElement" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2be9d5b6-cd77-411e-b841-3ee480311452" INVARIANT_UUID = "622f3043-48cc-4a54-a931-4b338101b3a9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2e88610e-aba3-4d43-84f2-bba006643ae0.assettoscatemplate" CURR_ARTIFACT_UUID = "2e88610e-aba3-4d43-84f2-bba006643ae0.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Automationcompositionelement-template.yml,0,2e88610e-aba3-4d43-84f2-bba006643ae0.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=c192d11e-1599-4fe9-9c17-4c8eee5d6bab TargetServiceName=null EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd ErrorCode=900 Received less parameters than expected for error with messageId SVC4063, expected: 1, actual: 0. Missing parameters are padded with null values. 2022-04-26T06:27:10.539Z [qtp2085002312-20] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "AutomationCompositionElement" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2be9d5b6-cd77-411e-b841-3ee480311452" INVARIANT_UUID = "622f3043-48cc-4a54-a931-4b338101b3a9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2e88610e-aba3-4d43-84f2-bba006643ae0.assettoscatemplate" CURR_ARTIFACT_UUID = "2e88610e-aba3-4d43-84f2-bba006643ae0.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Automationcompositionelement-template.yml,0,2e88610e-aba3-4d43-84f2-bba006643ae0.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Change LifecycleState - Certify ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=c192d11e-1599-4fe9-9c17-4c8eee5d6bab TargetServiceName=null EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd ErrorCode=500 Unexpected error during operation 2022-04-26T06:27:10.540Z [qtp2085002312-20] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "AutomationCompositionElement" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2be9d5b6-cd77-411e-b841-3ee480311452" INVARIANT_UUID = "622f3043-48cc-4a54-a931-4b338101b3a9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2e88610e-aba3-4d43-84f2-bba006643ae0.assettoscatemplate" CURR_ARTIFACT_UUID = "2e88610e-aba3-4d43-84f2-bba006643ae0.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Automationcompositionelement-template.yml,0,2e88610e-aba3-4d43-84f2-bba006643ae0.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Import Resource null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=c192d11e-1599-4fe9-9c17-4c8eee5d6bab TargetServiceName=null EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd ErrorCode=500 Unexpected error during operation 2022-04-26T06:27:10.544Z [qtp2085002312-20] ERROR o.o.s.b.s.ResourceUploadServlet AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "AutomationCompositionElement" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "" INVARIANT_UUID = "" PREV_VERSION = "1.0" CURR_VERSION = "" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "" STATUS = "404" DESC = "SVC4063: Error: Requested 'null' resource was not found." TOSCA_NODE_TYPE = "" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Import Resource null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=c192d11e-1599-4fe9-9c17-4c8eee5d6bab TargetServiceName=null EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd ErrorCode=500 #bulkImport('NodeTypesMetadataList(nodeMetadataList=[NodeTypeMetadata(name=Root, toscaName=tosca.nodes.Root, description=The TOSCA Node Type all other TOSCA base Node Types derive from, contactId=jh0003, resourceIconPath=defaulticon, model=AUTOMATION COMPOSITION, icon=null, vendorName=null, vendorRelease=null, resourceVendorModelNumber=null, resourceType=VFC, tags=[Root, tosca.nodes.Root], categories=[CategoryDefinition(super=CategoryDataDefinition(name=Generic, displayName=null, normalizedName=null, models=null, uniqueId=null, icons=null, useServiceSubstitutionForNestedServices=false, metadataKeys=null), subcategories=[SubCategoryDefinition(super=SubCategoryDataDefinition(name=Abstract, normalizedName=null, uniqueId=null, icons=null, metadataKeys=null), groupings=null)])]), NodeTypeMetadata(name=Participant, toscaName=org.onap.policy.clamp.acm.Participant, description=The TOSCA service template representing the Participant, contactId=jh0003, resourceIconPath=defaulticon, model=AUTOMATION COMPOSITION, icon=null, vendorName=null, vendorRelease=null, resourceVendorModelNumber=null, resourceType=VFC, tags=[Participant, org.onap.policy.clamp.acm.Participant], categories=[CategoryDefinition(super=CategoryDataDefinition(name=Generic, displayName=null, normalizedName=null, models=null, uniqueId=null, icons=null, useServiceSubstitutionForNestedServices=false, metadataKeys=null), subcategories=[SubCategoryDefinition(super=SubCategoryDataDefinition(name=Infrastructure, normalizedName=null, uniqueId=null, icons=null, metadataKeys=null), groupings=null)])]), NodeTypeMetadata(name=AutomationCompositionElement, toscaName=org.onap.policy.clamp.acm.AutomationCompositionElement, description=The TOSCA service template representing the AutomationCompositionElement, contactId=jh0003, resourceIconPath=defaulticon, model=AUTOMATION COMPOSITION, icon=null, vendorName=null, vendorRelease=null, resourceVendorModelNumber=null, resourceType=VFC, tags=[AutomationCompositionElement, org.onap.policy.clamp.acm.AutomationCompositionElement], categories=[CategoryDefinition(super=CategoryDataDefinition(name=Generic, displayName=null, normalizedName=null, models=null, uniqueId=null, icons=null, useServiceSubstitutionForNestedServices=false, metadataKeys=null), subcategories=[SubCategoryDefinition(super=SubCategoryDataDefinition(name=Infrastructure, normalizedName=null, uniqueId=null, icons=null, metadataKeys=null), groupings=null)])]), NodeTypeMetadata(name=AutomationComposition, toscaName=org.onap.policy.clamp.acm.AutomationComposition, description=The TOSCA service template representing the AutomationComposition, contactId=jh0003, resourceIconPath=defaulticon, model=AUTOMATION COMPOSITION, icon=null, vendorName=null, vendorRelease=null, resourceVendorModelNumber=null, resourceType=VFC, tags=[AutomationComposition, org.onap.policy.clamp.acm.AutomationComposition], categories=[CategoryDefinition(super=CategoryDataDefinition(name=Generic, displayName=null, normalizedName=null, models=null, uniqueId=null, icons=null, useServiceSubstitutionForNestedServices=false, metadataKeys=null), subcategories=[SubCategoryDefinition(super=SubCategoryDataDefinition(name=Infrastructure, normalizedName=null, uniqueId=null, icons=null, metadataKeys=null), groupings=null)])]), NodeTypeMetadata(name=PolicyAutomationCompositionElement, toscaName=org.onap.policy.clamp.acm.PolicyAutomationCompositionElement, description=The TOSCA service template representing the PolicyAutomationCompositionElement, contactId=jh0003, resourceIconPath=defaulticon, model=AUTOMATION COMPOSITION, icon=null, vendorName=null, vendorRelease=null, resourceVendorModelNumber=null, resourceType=VFC, tags=[PolicyAutomationCompositionElement, org.onap.policy.clamp.acm.PolicyAutomationCompositionElement], categories=[CategoryDefinition(super=CategoryDataDefinition(name=Generic, displayName=null, normalizedName=null, models=null, uniqueId=null, icons=null, useServiceSubstitutionForNestedServices=false, metadataKeys=null), subcategories=[SubCategoryDefinition(super=SubCategoryDataDefinition(name=Infrastructure, normalizedName=null, uniqueId=null, icons=null, metadataKeys=null), groupings=null)])]), NodeTypeMetadata(name=CDSAutomationCompositionElement, toscaName=org.onap.policy.clamp.acm.CDSAutomationCompositionElement, description=The TOSCA service template representing the CDSAutomationCompositionElement, contactId=jh0003, resourceIconPath=defaulticon, model=AUTOMATION COMPOSITION, icon=null, vendorName=null, vendorRelease=null, resourceVendorModelNumber=null, resourceType=VFC, tags=[CDSAutomationCompositionElement, org.onap.policy.clamp.acm.CDSAutomationCompositionElement], categories=[CategoryDefinition(super=CategoryDataDefinition(name=Generic, displayName=null, normalizedName=null, models=null, uniqueId=null, icons=null, useServiceSubstitutionForNestedServices=false, metadataKeys=null), subcategories=[SubCategoryDefinition(super=SubCategoryDataDefinition(name=Infrastructure, normalizedName=null, uniqueId=null, icons=null, metadataKeys=null), groupings=null)])]), NodeTypeMetadata(name=K8SMicroserviceAutomationCompositionElement, toscaName=org.onap.policy.clamp.acm.K8SMicroserviceAutomationCompositionElement, description=The TOSCA service template representing the K8SMicroserviceAutomationCompositionElement, contactId=jh0003, resourceIconPath=defaulticon, model=AUTOMATION COMPOSITION, icon=null, vendorName=null, vendorRelease=null, resourceVendorModelNumber=null, resourceType=VFC, tags=[K8SMicroserviceAutomationCompositionElement, org.onap.policy.clamp.acm.K8SMicroserviceAutomationCompositionElement], categories=[CategoryDefinition(super=CategoryDataDefinition(name=Generic, displayName=null, normalizedName=null, models=null, uniqueId=null, icons=null, useServiceSubstitutionForNestedServices=false, metadataKeys=null), subcategories=[SubCategoryDefinition(super=SubCategoryDataDefinition(name=Infrastructure, normalizedName=null, uniqueId=null, icons=null, metadataKeys=null), groupings=null)])]), NodeTypeMetadata(name=HttpAutomationCompositionElement, toscaName=org.onap.policy.clamp.acm.HttpAutomationCompositionElement, description=The TOSCA service template representing the HttpAutomationCompositionElement, contactId=jh0003, resourceIconPath=defaulticon, model=AUTOMATION COMPOSITION, icon=null, vendorName=null, vendorRelease=null, resourceVendorModelNumber=null, resourceType=VFC, tags=[HttpAutomationCompositionElement, org.onap.policy.clamp.acm.HttpAutomationCompositionElement], categories=[CategoryDefinition(super=CategoryDataDefinition(name=Generic, displayName=null, normalizedName=null, models=null, uniqueId=null, icons=null, useServiceSubstitutionForNestedServices=false, metadataKeys=null), subcategories=[SubCategoryDefinition(super=SubCategoryDataDefinition(name=Infrastructure, normalizedName=null, uniqueId=null, icons=null, metadataKeys=null), groupings=null)])])])', org.jvnet.mimepull.DataHead$ReadMultiStream@765f8211, false, 'jh0003', 'Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/catalog/upload/resource/import)@4e1a4d00'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic#lambda$decodeToscaArtifactPayload$5[377] in 18s 2022-04-26T06:27:10.545Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "AutomationCompositionElement" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "" INVARIANT_UUID = "" PREV_VERSION = "1.0" CURR_VERSION = "" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "" STATUS = "404" DESC = "SVC4063: Error: Requested 'null' resource was not found." TOSCA_NODE_TYPE = "" SupportablityAction=GENERATE CSAR ResponseCode=404 RequestID=null ElapsedTime=18498 PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=ERROR TargetEntity=Import Resource null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=c192d11e-1599-4fe9-9c17-4c8eee5d6bab TargetServiceName=null ErrorDesc=Not Found EntryTimestamp=2022-04-26T06:26:52.047Z InvocationID=a7283827-9627-4a39-8c16-07c2c9cfb8fd ErrorCode=404 LogTimestamp=2022-04-26T06:27:10.545462Z ResponseDesc=Not Found Exiting. 2022-04-26T06:27:10.546Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/resource/import HTTP/1.1 SC="404" 2022-04-26T06:27:10.554Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=7273215d-df54-4115-adef-f32d5a773caa InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS RemoteHost=10.233.69.251 ServiceName=v1/catalog/lock ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:27:10.554491Z InvocationID=48593ba4-e785-44cc-968d-2c60b0af4323 LogTimestamp=2022-04-26T06:27:10.554522Z Entering 2022-04-26T06:27:10.554Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=7273215d-df54-4115-adef-f32d5a773caa InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.69.251 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.69.251 ServiceName=v1/catalog/lock ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:27:10.554491Z InvocationID=48593ba4-e785-44cc-968d-2c60b0af4323 LogTimestamp=2022-04-26T06:27:10.554522Z localAddr=10.233.65.41 remoteAddr=10.233.69.251 POST /sdc2/rest/v1/catalog/lock HTTP/1.1 2022-04-26T06:27:10.556Z [qtp2085002312-18] INFO o.o.sdc.be.servlets.LockServlet RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/lock EntryTimestamp=2022-04-26T06:27:10.555Z InvocationID=68043e7f-99a8-490a-bdee-ac786023ca02 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb User jh0003 attempting to set disable locking with value false 2022-04-26T06:27:10.556Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/lock EntryTimestamp=2022-04-26T06:27:10.555Z InvocationID=68043e7f-99a8-490a-bdee-ac786023ca02 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Toggling disable locking from true to false 2022-04-26T06:27:10.556Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=1 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/lock EntryTimestamp=2022-04-26T06:27:10.555Z InvocationID=68043e7f-99a8-490a-bdee-ac786023ca02 LogTimestamp=2022-04-26T06:27:10.556710Z ResponseDesc=OK Exiting. 2022-04-26T06:27:10.557Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/lock HTTP/1.1 SC="200" 2022-04-26T06:27:10.893Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=dc5457ad-b5f6-4ba4-872a-4b0e10f40418 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:27:10.894Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=dc5457ad-b5f6-4ba4-872a-4b0e10f40418 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:27:11.955Z [cluster2-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:27:11.956Z [cluster2-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 30000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:27:12.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37e8ee14-aad8-40f4-94b7-1b3dbc211bb9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:27:12.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37e8ee14-aad8-40f4-94b7-1b3dbc211bb9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@47fa80ec 2022-04-26T06:27:12.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37e8ee14-aad8-40f4-94b7-1b3dbc211bb9 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:27:12.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37e8ee14-aad8-40f4-94b7-1b3dbc211bb9 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:27:13.901Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=37e8ee14-aad8-40f4-94b7-1b3dbc211bb9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:27:13.901Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=37e8ee14-aad8-40f4-94b7-1b3dbc211bb9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:27:15.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=127f05fd-172d-4e77-ac4d-0a4d3d090d79 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:27:15.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=127f05fd-172d-4e77-ac4d-0a4d3d090d79 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5b46b458 2022-04-26T06:27:15.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=127f05fd-172d-4e77-ac4d-0a4d3d090d79 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:27:15.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=127f05fd-172d-4e77-ac4d-0a4d3d090d79 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:27:16.813Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=127f05fd-172d-4e77-ac4d-0a4d3d090d79 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:27:16.813Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=127f05fd-172d-4e77-ac4d-0a4d3d090d79 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:27:18.796Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c34e28c-acbd-4521-826e-74893b7b499f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:27:18.796Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c34e28c-acbd-4521-826e-74893b7b499f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@42b9ea95 2022-04-26T06:27:18.796Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c34e28c-acbd-4521-826e-74893b7b499f PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:27:18.796Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c34e28c-acbd-4521-826e-74893b7b499f PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:27:19.865Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=4c34e28c-acbd-4521-826e-74893b7b499f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:27:19.865Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=4c34e28c-acbd-4521-826e-74893b7b499f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:27:20.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:27:20.377Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:27:22.538Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ce62874-063c-4277-be00-66cd903a89d6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:27:22.538Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ce62874-063c-4277-be00-66cd903a89d6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7b2f09b4 2022-04-26T06:27:22.538Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ce62874-063c-4277-be00-66cd903a89d6 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:27:22.538Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ce62874-063c-4277-be00-66cd903a89d6 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:27:23.565Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=9ce62874-063c-4277-be00-66cd903a89d6 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:27:23.565Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=9ce62874-063c-4277-be00-66cd903a89d6 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:27:25.338Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=38ede4ba-8fb6-4cb5-91dd-77ace9b93d4c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:27:25.339Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=38ede4ba-8fb6-4cb5-91dd-77ace9b93d4c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@189be46f 2022-04-26T06:27:25.339Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=38ede4ba-8fb6-4cb5-91dd-77ace9b93d4c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:27:25.339Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=38ede4ba-8fb6-4cb5-91dd-77ace9b93d4c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:27:26.384Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=38ede4ba-8fb6-4cb5-91dd-77ace9b93d4c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:27:26.384Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=38ede4ba-8fb6-4cb5-91dd-77ace9b93d4c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:27:28.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c94019c1-159c-4e59-86d2-f5407fdcfb0c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:27:28.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c94019c1-159c-4e59-86d2-f5407fdcfb0c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@223f2f9d 2022-04-26T06:27:28.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c94019c1-159c-4e59-86d2-f5407fdcfb0c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:27:28.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c94019c1-159c-4e59-86d2-f5407fdcfb0c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:27:29.485Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c94019c1-159c-4e59-86d2-f5407fdcfb0c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:27:29.485Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c94019c1-159c-4e59-86d2-f5407fdcfb0c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:27:31.342Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bed36752-f4ad-48dc-96b8-1160e60a1ff8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:27:31.342Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bed36752-f4ad-48dc-96b8-1160e60a1ff8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2a3c5067 2022-04-26T06:27:31.342Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bed36752-f4ad-48dc-96b8-1160e60a1ff8 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:27:31.342Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bed36752-f4ad-48dc-96b8-1160e60a1ff8 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:27:32.367Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=bed36752-f4ad-48dc-96b8-1160e60a1ff8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:27:32.367Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=bed36752-f4ad-48dc-96b8-1160e60a1ff8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:27:34.333Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ad00bed-c842-4209-8851-e5b16775c5c4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:27:34.333Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ad00bed-c842-4209-8851-e5b16775c5c4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@27292d7f 2022-04-26T06:27:34.333Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ad00bed-c842-4209-8851-e5b16775c5c4 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:27:34.334Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ad00bed-c842-4209-8851-e5b16775c5c4 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:27:35.370Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:27:35.373Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=8ad00bed-c842-4209-8851-e5b16775c5c4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:27:35.373Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=8ad00bed-c842-4209-8851-e5b16775c5c4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:27:35.379Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:27:37.632Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cf46f56-7262-4874-bb93-2a555fe044fb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:27:37.632Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cf46f56-7262-4874-bb93-2a555fe044fb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@19a2679a 2022-04-26T06:27:37.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cf46f56-7262-4874-bb93-2a555fe044fb PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:27:37.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cf46f56-7262-4874-bb93-2a555fe044fb PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:27:38.665Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=0cf46f56-7262-4874-bb93-2a555fe044fb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:27:38.666Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=0cf46f56-7262-4874-bb93-2a555fe044fb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:27:39.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2d3dc237-3651-4054-b029-aa9ab4bb89f9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:27:39.794Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2d3dc237-3651-4054-b029-aa9ab4bb89f9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@67b5413 2022-04-26T06:27:39.794Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2d3dc237-3651-4054-b029-aa9ab4bb89f9 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:27:39.794Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2d3dc237-3651-4054-b029-aa9ab4bb89f9 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:27:40.812Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=2d3dc237-3651-4054-b029-aa9ab4bb89f9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:27:40.813Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=2d3dc237-3651-4054-b029-aa9ab4bb89f9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:27:42.838Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08c7203e-2d95-430e-81f6-c59698bc1145 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:27:42.838Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08c7203e-2d95-430e-81f6-c59698bc1145 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7013205c 2022-04-26T06:27:42.838Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08c7203e-2d95-430e-81f6-c59698bc1145 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:27:42.839Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08c7203e-2d95-430e-81f6-c59698bc1145 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:27:43.884Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=08c7203e-2d95-430e-81f6-c59698bc1145 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:27:43.885Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=08c7203e-2d95-430e-81f6-c59698bc1145 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:27:45.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=caecd218-c8d8-4ab4-8918-11730e45fedb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:27:45.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=caecd218-c8d8-4ab4-8918-11730e45fedb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7ae059b8 2022-04-26T06:27:45.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=caecd218-c8d8-4ab4-8918-11730e45fedb PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:27:45.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=caecd218-c8d8-4ab4-8918-11730e45fedb PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:27:46.832Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=caecd218-c8d8-4ab4-8918-11730e45fedb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:27:46.833Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=caecd218-c8d8-4ab4-8918-11730e45fedb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:27:47.258Z [cluster2-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:27:47.259Z [cluster2-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 30000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:27:48.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e782c5c-60e4-4803-a9ab-7273be7fcfa8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:27:48.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e782c5c-60e4-4803-a9ab-7273be7fcfa8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@51ec38d5 2022-04-26T06:27:48.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e782c5c-60e4-4803-a9ab-7273be7fcfa8 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:27:48.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e782c5c-60e4-4803-a9ab-7273be7fcfa8 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:27:49.805Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=1e782c5c-60e4-4803-a9ab-7273be7fcfa8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:27:49.806Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=1e782c5c-60e4-4803-a9ab-7273be7fcfa8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:27:50.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:27:50.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:27:52.454Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6187015b-c119-4438-9c1f-6f17ff51991b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:27:52.454Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6187015b-c119-4438-9c1f-6f17ff51991b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3597af48 2022-04-26T06:27:52.454Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6187015b-c119-4438-9c1f-6f17ff51991b PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:27:52.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6187015b-c119-4438-9c1f-6f17ff51991b PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:27:52.687Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:27:52.688Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:27:52.688Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:27:52.753Z [cluster1-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:27:52.754Z [cluster1-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 128000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:27:53.485Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=6187015b-c119-4438-9c1f-6f17ff51991b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:27:53.485Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=6187015b-c119-4438-9c1f-6f17ff51991b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:27:55.534Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8315055c-3a96-4518-8ff1-628d3349ce37 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:27:55.535Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8315055c-3a96-4518-8ff1-628d3349ce37 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1a74569f 2022-04-26T06:27:55.535Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8315055c-3a96-4518-8ff1-628d3349ce37 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:27:55.535Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8315055c-3a96-4518-8ff1-628d3349ce37 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:27:56.553Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=8315055c-3a96-4518-8ff1-628d3349ce37 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:27:56.553Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=8315055c-3a96-4518-8ff1-628d3349ce37 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:27:58.550Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90119cfc-fd71-4950-b934-7dbecb6b042b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:27:58.550Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90119cfc-fd71-4950-b934-7dbecb6b042b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4f981142 2022-04-26T06:27:58.550Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90119cfc-fd71-4950-b934-7dbecb6b042b PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:27:58.550Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90119cfc-fd71-4950-b934-7dbecb6b042b PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:27:59.561Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=90119cfc-fd71-4950-b934-7dbecb6b042b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:27:59.562Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=90119cfc-fd71-4950-b934-7dbecb6b042b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:28:01.534Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1498e86c-251c-4c7b-af9b-bfc3e20f3379 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:28:01.537Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1498e86c-251c-4c7b-af9b-bfc3e20f3379 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2a9f079a 2022-04-26T06:28:01.537Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1498e86c-251c-4c7b-af9b-bfc3e20f3379 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:28:01.537Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1498e86c-251c-4c7b-af9b-bfc3e20f3379 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:28:02.578Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=1498e86c-251c-4c7b-af9b-bfc3e20f3379 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:28:02.578Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=1498e86c-251c-4c7b-af9b-bfc3e20f3379 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:28:04.346Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec44d233-a668-4907-9ba2-575c0a2df202 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:28:04.346Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec44d233-a668-4907-9ba2-575c0a2df202 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@a08eff9 2022-04-26T06:28:04.346Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec44d233-a668-4907-9ba2-575c0a2df202 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:28:04.346Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec44d233-a668-4907-9ba2-575c0a2df202 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:28:05.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:28:05.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:28:05.385Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ec44d233-a668-4907-9ba2-575c0a2df202 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:28:05.386Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ec44d233-a668-4907-9ba2-575c0a2df202 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:28:07.335Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c5721dd-3ca4-479c-a874-632e52ecb910 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:28:07.335Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c5721dd-3ca4-479c-a874-632e52ecb910 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@71e87532 2022-04-26T06:28:07.335Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c5721dd-3ca4-479c-a874-632e52ecb910 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:28:07.335Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c5721dd-3ca4-479c-a874-632e52ecb910 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:28:07.658Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:28:08.365Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=5c5721dd-3ca4-479c-a874-632e52ecb910 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:28:08.365Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=5c5721dd-3ca4-479c-a874-632e52ecb910 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:28:09.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9982746-56e5-4167-8783-c882b53eecd5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:28:09.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9982746-56e5-4167-8783-c882b53eecd5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@168df983 2022-04-26T06:28:09.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9982746-56e5-4167-8783-c882b53eecd5 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:28:09.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9982746-56e5-4167-8783-c882b53eecd5 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:28:10.793Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e9982746-56e5-4167-8783-c882b53eecd5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:28:10.793Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e9982746-56e5-4167-8783-c882b53eecd5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:28:12.837Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1fe4168-3b6d-4e8e-a2c5-cc83c8deea53 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:28:12.837Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1fe4168-3b6d-4e8e-a2c5-cc83c8deea53 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4fa414c6 2022-04-26T06:28:12.837Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1fe4168-3b6d-4e8e-a2c5-cc83c8deea53 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:28:12.838Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1fe4168-3b6d-4e8e-a2c5-cc83c8deea53 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:28:13.865Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e1fe4168-3b6d-4e8e-a2c5-cc83c8deea53 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:28:13.866Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e1fe4168-3b6d-4e8e-a2c5-cc83c8deea53 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:28:15.798Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4413112c-ecd9-46bf-9dd1-7a5513da8a52 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:28:15.798Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4413112c-ecd9-46bf-9dd1-7a5513da8a52 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@50dcf4bd 2022-04-26T06:28:15.798Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4413112c-ecd9-46bf-9dd1-7a5513da8a52 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:28:15.799Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4413112c-ecd9-46bf-9dd1-7a5513da8a52 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:28:16.843Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=4413112c-ecd9-46bf-9dd1-7a5513da8a52 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:28:16.843Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=4413112c-ecd9-46bf-9dd1-7a5513da8a52 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:28:18.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0947215-f56d-4d6f-ab6c-6c87818e8d97 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:28:18.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0947215-f56d-4d6f-ab6c-6c87818e8d97 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f5b078e 2022-04-26T06:28:18.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0947215-f56d-4d6f-ab6c-6c87818e8d97 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:28:18.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0947215-f56d-4d6f-ab6c-6c87818e8d97 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:28:19.817Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f0947215-f56d-4d6f-ab6c-6c87818e8d97 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:28:19.817Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f0947215-f56d-4d6f-ab6c-6c87818e8d97 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:28:20.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:28:20.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:28:22.644Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5fb9334b-99b4-42c8-b714-2bf422f5b8bd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:28:22.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5fb9334b-99b4-42c8-b714-2bf422f5b8bd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@43f541e9 2022-04-26T06:28:22.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5fb9334b-99b4-42c8-b714-2bf422f5b8bd PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:28:22.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5fb9334b-99b4-42c8-b714-2bf422f5b8bd PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:28:23.690Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=5fb9334b-99b4-42c8-b714-2bf422f5b8bd ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:28:23.691Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=5fb9334b-99b4-42c8-b714-2bf422f5b8bd ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:28:23.756Z [cluster2-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:28:23.757Z [cluster2-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 30000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:28:25.348Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b2a9f949-bba6-4210-a789-51a7df578a28 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:28:25.348Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b2a9f949-bba6-4210-a789-51a7df578a28 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@45399ba8 2022-04-26T06:28:25.349Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b2a9f949-bba6-4210-a789-51a7df578a28 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:28:25.349Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b2a9f949-bba6-4210-a789-51a7df578a28 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:28:26.383Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=b2a9f949-bba6-4210-a789-51a7df578a28 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:28:26.384Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=b2a9f949-bba6-4210-a789-51a7df578a28 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:28:28.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36d6aae9-fa7a-4557-9196-cc6de60c3f45 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:28:28.440Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36d6aae9-fa7a-4557-9196-cc6de60c3f45 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@16a426d 2022-04-26T06:28:28.440Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36d6aae9-fa7a-4557-9196-cc6de60c3f45 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:28:28.440Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36d6aae9-fa7a-4557-9196-cc6de60c3f45 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:28:29.481Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=36d6aae9-fa7a-4557-9196-cc6de60c3f45 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:28:29.482Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=36d6aae9-fa7a-4557-9196-cc6de60c3f45 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:28:31.443Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4928c69-f1f7-4afb-aaf1-6e690a1dbe86 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:28:31.443Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4928c69-f1f7-4afb-aaf1-6e690a1dbe86 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2668de3a 2022-04-26T06:28:31.444Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4928c69-f1f7-4afb-aaf1-6e690a1dbe86 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:28:31.444Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4928c69-f1f7-4afb-aaf1-6e690a1dbe86 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:28:32.461Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=a4928c69-f1f7-4afb-aaf1-6e690a1dbe86 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:28:32.462Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=a4928c69-f1f7-4afb-aaf1-6e690a1dbe86 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:28:34.251Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd2e6416-c25b-45c8-a658-5b43638f7afe PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:28:34.251Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd2e6416-c25b-45c8-a658-5b43638f7afe PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@507a2864 2022-04-26T06:28:34.251Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd2e6416-c25b-45c8-a658-5b43638f7afe PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:28:34.251Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd2e6416-c25b-45c8-a658-5b43638f7afe PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:28:35.273Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=fd2e6416-c25b-45c8-a658-5b43638f7afe ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:28:35.273Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=fd2e6416-c25b-45c8-a658-5b43638f7afe ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:28:35.369Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:28:35.396Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:28:37.528Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d792658-c11e-479f-9bbc-ca8a45cf31f7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:28:37.528Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d792658-c11e-479f-9bbc-ca8a45cf31f7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@30a3b341 2022-04-26T06:28:37.528Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d792658-c11e-479f-9bbc-ca8a45cf31f7 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:28:37.528Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d792658-c11e-479f-9bbc-ca8a45cf31f7 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:28:38.569Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=9d792658-c11e-479f-9bbc-ca8a45cf31f7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:28:38.570Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=9d792658-c11e-479f-9bbc-ca8a45cf31f7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:28:39.788Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c36664d-3ecb-40b2-b2a3-f3f6cc5c1bb9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:28:39.788Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c36664d-3ecb-40b2-b2a3-f3f6cc5c1bb9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@31112bd6 2022-04-26T06:28:39.789Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c36664d-3ecb-40b2-b2a3-f3f6cc5c1bb9 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:28:39.789Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c36664d-3ecb-40b2-b2a3-f3f6cc5c1bb9 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:28:40.813Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=1c36664d-3ecb-40b2-b2a3-f3f6cc5c1bb9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:28:40.813Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=1c36664d-3ecb-40b2-b2a3-f3f6cc5c1bb9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:28:42.792Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ca3bf06-4085-4d6c-b883-9284e04f4a4d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:28:42.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ca3bf06-4085-4d6c-b883-9284e04f4a4d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1dad4eb9 2022-04-26T06:28:42.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ca3bf06-4085-4d6c-b883-9284e04f4a4d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:28:42.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ca3bf06-4085-4d6c-b883-9284e04f4a4d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:28:43.821Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=6ca3bf06-4085-4d6c-b883-9284e04f4a4d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:28:43.822Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=6ca3bf06-4085-4d6c-b883-9284e04f4a4d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:28:45.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c3c733de-a386-4855-8f02-6f00764d6eac PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:28:45.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c3c733de-a386-4855-8f02-6f00764d6eac PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5d6e3da9 2022-04-26T06:28:45.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c3c733de-a386-4855-8f02-6f00764d6eac PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:28:45.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c3c733de-a386-4855-8f02-6f00764d6eac PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:28:46.793Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c3c733de-a386-4855-8f02-6f00764d6eac ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:28:46.793Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c3c733de-a386-4855-8f02-6f00764d6eac ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:28:48.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29835c40-5c58-4e6f-a8f9-055bc503c7a0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:28:48.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29835c40-5c58-4e6f-a8f9-055bc503c7a0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@54d4fd6a 2022-04-26T06:28:48.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29835c40-5c58-4e6f-a8f9-055bc503c7a0 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:28:48.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29835c40-5c58-4e6f-a8f9-055bc503c7a0 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:28:49.801Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=29835c40-5c58-4e6f-a8f9-055bc503c7a0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:28:49.801Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=29835c40-5c58-4e6f-a8f9-055bc503c7a0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:28:50.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:28:50.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:28:52.539Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d350ccf1-2ae3-4f75-8245-74149737d006 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:28:52.539Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d350ccf1-2ae3-4f75-8245-74149737d006 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5262706f 2022-04-26T06:28:52.540Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d350ccf1-2ae3-4f75-8245-74149737d006 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:28:52.540Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d350ccf1-2ae3-4f75-8245-74149737d006 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:28:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:28:52.687Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:28:52.687Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:28:53.577Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=d350ccf1-2ae3-4f75-8245-74149737d006 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:28:53.578Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=d350ccf1-2ae3-4f75-8245-74149737d006 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:28:55.443Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=92c1a2cd-81da-4a68-926a-411cf021c644 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:28:55.443Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=92c1a2cd-81da-4a68-926a-411cf021c644 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5f12c832 2022-04-26T06:28:55.444Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=92c1a2cd-81da-4a68-926a-411cf021c644 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:28:55.444Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=92c1a2cd-81da-4a68-926a-411cf021c644 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:28:56.458Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=92c1a2cd-81da-4a68-926a-411cf021c644 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:28:56.460Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=92c1a2cd-81da-4a68-926a-411cf021c644 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:28:58.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f504fd67-a753-4f7e-8608-25fda247b1c0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:28:58.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f504fd67-a753-4f7e-8608-25fda247b1c0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@c267c34 2022-04-26T06:28:58.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f504fd67-a753-4f7e-8608-25fda247b1c0 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:28:58.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f504fd67-a753-4f7e-8608-25fda247b1c0 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:28:59.157Z [cluster2-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:28:59.159Z [cluster2-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 30000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:28:59.465Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f504fd67-a753-4f7e-8608-25fda247b1c0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:28:59.466Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f504fd67-a753-4f7e-8608-25fda247b1c0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:29:01.531Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1741cd06-5e4b-4ad1-845a-3574d4b35372 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:29:01.532Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1741cd06-5e4b-4ad1-845a-3574d4b35372 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1bc766c0 2022-04-26T06:29:01.532Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1741cd06-5e4b-4ad1-845a-3574d4b35372 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:29:01.532Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1741cd06-5e4b-4ad1-845a-3574d4b35372 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:29:02.573Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=1741cd06-5e4b-4ad1-845a-3574d4b35372 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:29:02.573Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=1741cd06-5e4b-4ad1-845a-3574d4b35372 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:29:04.337Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3927356e-75ae-4f7b-950a-cd951f5731b4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:29:04.339Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3927356e-75ae-4f7b-950a-cd951f5731b4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5da717fc 2022-04-26T06:29:04.339Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3927356e-75ae-4f7b-950a-cd951f5731b4 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:29:04.340Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3927356e-75ae-4f7b-950a-cd951f5731b4 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:29:05.361Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3927356e-75ae-4f7b-950a-cd951f5731b4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:29:05.362Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3927356e-75ae-4f7b-950a-cd951f5731b4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:29:05.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:29:05.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:29:07.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64e6b476-4d96-4060-8efe-526794d4e2a1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:29:07.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64e6b476-4d96-4060-8efe-526794d4e2a1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@44ad1a7f 2022-04-26T06:29:07.435Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64e6b476-4d96-4060-8efe-526794d4e2a1 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:29:07.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64e6b476-4d96-4060-8efe-526794d4e2a1 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:29:07.714Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:29:08.457Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=64e6b476-4d96-4060-8efe-526794d4e2a1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:29:08.457Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=64e6b476-4d96-4060-8efe-526794d4e2a1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:29:09.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ca1606ef-8ff5-4bce-8b7a-63cb3b5fe95b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:29:09.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ca1606ef-8ff5-4bce-8b7a-63cb3b5fe95b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3b17438b 2022-04-26T06:29:09.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ca1606ef-8ff5-4bce-8b7a-63cb3b5fe95b PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:29:09.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ca1606ef-8ff5-4bce-8b7a-63cb3b5fe95b PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:29:10.828Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ca1606ef-8ff5-4bce-8b7a-63cb3b5fe95b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:29:10.829Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ca1606ef-8ff5-4bce-8b7a-63cb3b5fe95b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:29:12.789Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe9d4bc4-e0b7-4f74-b520-9d33e9808c32 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:29:12.790Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe9d4bc4-e0b7-4f74-b520-9d33e9808c32 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@17b388ef 2022-04-26T06:29:12.790Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe9d4bc4-e0b7-4f74-b520-9d33e9808c32 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:29:12.790Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe9d4bc4-e0b7-4f74-b520-9d33e9808c32 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:29:13.833Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=fe9d4bc4-e0b7-4f74-b520-9d33e9808c32 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:29:13.833Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=fe9d4bc4-e0b7-4f74-b520-9d33e9808c32 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:29:15.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad481311-e188-4f7c-b715-ba7f2e66a849 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:29:15.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad481311-e188-4f7c-b715-ba7f2e66a849 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1d620696 2022-04-26T06:29:15.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad481311-e188-4f7c-b715-ba7f2e66a849 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:29:15.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad481311-e188-4f7c-b715-ba7f2e66a849 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:29:16.816Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ad481311-e188-4f7c-b715-ba7f2e66a849 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:29:16.817Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ad481311-e188-4f7c-b715-ba7f2e66a849 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:29:17.107Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=782b9364-0158-454d-9201-d4456b351e7d InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServiceName=v1/user/jh0003 ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:29:17.106215Z InvocationID=d3a4f3da-0cf1-4698-80b1-9184b052e2c3 LogTimestamp=2022-04-26T06:29:17.106432Z Entering 2022-04-26T06:29:17.107Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=782b9364-0158-454d-9201-d4456b351e7d InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServiceName=v1/user/jh0003 ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:29:17.106215Z InvocationID=d3a4f3da-0cf1-4698-80b1-9184b052e2c3 LogTimestamp=2022-04-26T06:29:17.106432Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2022-04-26T06:29:17.112Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=4 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/user/jh0003 EntryTimestamp=2022-04-26T06:29:17.108Z InvocationID=dd24da16-242d-4656-8bc5-2ff4e994186b LogTimestamp=2022-04-26T06:29:17.112614Z ResponseDesc=OK Exiting. 2022-04-26T06:29:17.113Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2022-04-26T06:29:17.122Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=119250 PartnerName=kube-probe/1.22 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=ERROR ServerIPAddress=10.233.65.41 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2022-04-26T06:27:17.872Z InvocationID=61a3b253-f3a2-414d-ad68-72d20ea65aae ErrorCode=500 LogTimestamp=2022-04-26T06:29:17.122305Z ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:29:17.123Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdemo HTTP/1.1 SC="500" 2022-04-26T06:29:17.132Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=7a7290ab-fb90-44e3-bbfe-185abd50899f InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2022-04-26T06:29:17.132239Z InvocationID=05bbe300-bea8-46a1-96ac-0bafa41993ab LogTimestamp=2022-04-26T06:29:17.132299Z Entering 2022-04-26T06:29:17.132Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=7a7290ab-fb90-44e3-bbfe-185abd50899f InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2022-04-26T06:29:17.132239Z InvocationID=05bbe300-bea8-46a1-96ac-0bafa41993ab LogTimestamp=2022-04-26T06:29:17.132299Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/user HTTP/1.1 2022-04-26T06:29:17.152Z [qtp2085002312-16] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'demo' ID already exists." RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/user EntryTimestamp=2022-04-26T06:29:17.133Z InvocationID=35ed37a4-4126-46a7-a8c2-6e2194c5815c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@2f54ec8f', 'User [firstName=demo, lastName=demo, userId=demo, email=demo@openecomp.org, role=ADMIN, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1119] in 16ms 2022-04-26T06:29:17.154Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'demo' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=21 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2022-04-26T06:29:17.133Z InvocationID=35ed37a4-4126-46a7-a8c2-6e2194c5815c ErrorCode=409 LogTimestamp=2022-04-26T06:29:17.154434Z ResponseDesc=Conflict Exiting. 2022-04-26T06:29:17.155Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2022-04-26T06:29:17.161Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=59284 PartnerName=kube-probe/1.22 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=ERROR ServerIPAddress=10.233.65.41 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2022-04-26T06:28:17.876Z InvocationID=cb0d698e-c47b-4f58-8e0a-3c75535e30ce ErrorCode=500 LogTimestamp=2022-04-26T06:29:17.160893Z ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:29:17.161Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usergv0001 HTTP/1.1 SC="500" 2022-04-26T06:29:17.167Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=44d4d288-3037-45aa-99ec-9349573c008d InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServiceName=v1/user ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:29:17.167420Z InvocationID=1769755f-6f36-4113-b463-21fa9659981a LogTimestamp=2022-04-26T06:29:17.167483Z Entering 2022-04-26T06:29:17.167Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=44d4d288-3037-45aa-99ec-9349573c008d InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServiceName=v1/user ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:29:17.167420Z InvocationID=1769755f-6f36-4113-b463-21fa9659981a LogTimestamp=2022-04-26T06:29:17.167483Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/user HTTP/1.1 2022-04-26T06:29:17.173Z [qtp2085002312-15] ERROR o.o.sdc.be.servlets.UserAdminServlet RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/user EntryTimestamp=2022-04-26T06:29:17.168Z InvocationID=b10b41cf-d879-4195-9070-6902bd25ae1a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@2f54ec8f', 'User [firstName=Giuseppe, lastName=Verdi, userId=gv0001, email=gv0001@openecomp.org, role=GOVERNOR, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.user.UserBusinessLogic#validateRole[183] in 4ms 2022-04-26T06:29:17.175Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=400 RequestID=null ElapsedTime=6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/user ErrorDesc=Bad Request EntryTimestamp=2022-04-26T06:29:17.168Z InvocationID=b10b41cf-d879-4195-9070-6902bd25ae1a ErrorCode=400 LogTimestamp=2022-04-26T06:29:17.174918Z ResponseDesc=Bad Request Exiting. 2022-04-26T06:29:17.175Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="400" 2022-04-26T06:29:17.181Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=59309 PartnerName=kube-probe/1.22 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=ERROR ServerIPAddress=10.233.65.41 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2022-04-26T06:28:17.872Z InvocationID=bd53819d-1f9e-4f05-b9c8-390dde498c3c ErrorCode=500 LogTimestamp=2022-04-26T06:29:17.181307Z ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:29:17.182Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjh0003 HTTP/1.1 SC="500" 2022-04-26T06:29:17.190Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=381c384c-227a-4488-a1e9-f8bb8763fec2 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServiceName=v1/user ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:29:17.189825Z InvocationID=e88e07e0-6d45-4999-9362-49fd7e305b28 LogTimestamp=2022-04-26T06:29:17.189906Z Entering 2022-04-26T06:29:17.191Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=381c384c-227a-4488-a1e9-f8bb8763fec2 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServiceName=v1/user ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:29:17.189825Z InvocationID=e88e07e0-6d45-4999-9362-49fd7e305b28 LogTimestamp=2022-04-26T06:29:17.189906Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/user HTTP/1.1 2022-04-26T06:29:17.199Z [qtp2085002312-13] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/user EntryTimestamp=2022-04-26T06:29:17.192Z InvocationID=50f5dc15-3afb-414e-bebc-6a5bbdd60f8c PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@2f54ec8f', 'User [firstName=Jimmy, lastName=Hendrix, userId=jh0003, email=jh0003@openecomp.org, role=Admin, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1119] in 4ms 2022-04-26T06:29:17.200Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." ResponseCode=409 RequestID=null ElapsedTime=8 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2022-04-26T06:29:17.192Z InvocationID=50f5dc15-3afb-414e-bebc-6a5bbdd60f8c ErrorCode=409 LogTimestamp=2022-04-26T06:29:17.200040Z ResponseDesc=Conflict Exiting. 2022-04-26T06:29:17.200Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2022-04-26T06:29:17.206Z [qtp2085002312-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:29:17.206528Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:29:17.206Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:29:17.206528Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:29:17.207Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjm0007 HTTP/1.1 SC="500" 2022-04-26T06:29:17.212Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=e149cf70-75a0-4ccd-b3d2-a2c2782c522b InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServiceName=v1/user ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:29:17.212034Z InvocationID=8471251a-cfa3-40f0-921a-7b003c8e573d LogTimestamp=2022-04-26T06:29:17.212071Z Entering 2022-04-26T06:29:17.212Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=e149cf70-75a0-4ccd-b3d2-a2c2782c522b InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServiceName=v1/user ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:29:17.212034Z InvocationID=8471251a-cfa3-40f0-921a-7b003c8e573d LogTimestamp=2022-04-26T06:29:17.212071Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/user HTTP/1.1 2022-04-26T06:29:17.223Z [qtp2085002312-18] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'jm0007' ID already exists." RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/user EntryTimestamp=2022-04-26T06:29:17.212Z InvocationID=8c2eb850-8127-47fe-82cf-f2be8cb96a18 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@2f54ec8f', 'User [firstName=Joni, lastName=Mitchell, userId=jm0007, email=jm0007@openecomp.org, role=TESTER, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1119] in 9ms 2022-04-26T06:29:17.224Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'jm0007' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=12 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2022-04-26T06:29:17.212Z InvocationID=8c2eb850-8127-47fe-82cf-f2be8cb96a18 ErrorCode=409 LogTimestamp=2022-04-26T06:29:17.224332Z ResponseDesc=Conflict Exiting. 2022-04-26T06:29:17.224Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2022-04-26T06:29:17.229Z [qtp2085002312-16] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:29:17.229774Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:29:17.230Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:29:17.229774Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:29:17.230Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercs0008 HTTP/1.1 SC="500" 2022-04-26T06:29:17.236Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c3e1f47a-f9be-4071-9713-614a08fe30f1 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServiceName=v1/user ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:29:17.236233Z InvocationID=83a3afde-f35e-4fd5-9160-76b061cc9088 LogTimestamp=2022-04-26T06:29:17.236284Z Entering 2022-04-26T06:29:17.236Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c3e1f47a-f9be-4071-9713-614a08fe30f1 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServiceName=v1/user ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:29:17.236233Z InvocationID=83a3afde-f35e-4fd5-9160-76b061cc9088 LogTimestamp=2022-04-26T06:29:17.236284Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/user HTTP/1.1 2022-04-26T06:29:17.246Z [qtp2085002312-14] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'cs0008' ID already exists." RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/user EntryTimestamp=2022-04-26T06:29:17.237Z InvocationID=9ed1ce32-1a01-453a-a743-26b1faa275dd PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@2f54ec8f', 'User [firstName=Carlos, lastName=Santana, userId=cs0008, email=cs0008r@openecomp.org, role=DESIGNER, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1119] in 8ms 2022-04-26T06:29:17.247Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'cs0008' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2022-04-26T06:29:17.237Z InvocationID=9ed1ce32-1a01-453a-a743-26b1faa275dd ErrorCode=409 LogTimestamp=2022-04-26T06:29:17.246896Z ResponseDesc=Conflict Exiting. 2022-04-26T06:29:17.247Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2022-04-26T06:29:17.402Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=071a7b6d-0393-4e2e-a4ec-8ba2617a2708 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServiceName=v1/user/jh0003 ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:29:17.401784Z InvocationID=0f3019c4-f6e0-44ad-b34d-dab507f7edd4 LogTimestamp=2022-04-26T06:29:17.401871Z Entering 2022-04-26T06:29:17.402Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=071a7b6d-0393-4e2e-a4ec-8ba2617a2708 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServiceName=v1/user/jh0003 ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:29:17.401784Z InvocationID=0f3019c4-f6e0-44ad-b34d-dab507f7edd4 LogTimestamp=2022-04-26T06:29:17.401871Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2022-04-26T06:29:17.404Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=2 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/user/jh0003 EntryTimestamp=2022-04-26T06:29:17.402Z InvocationID=771d6c47-22c7-4250-95fd-6fd8f1e536a8 LogTimestamp=2022-04-26T06:29:17.404618Z ResponseDesc=OK Exiting. 2022-04-26T06:29:17.405Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2022-04-26T06:29:17.411Z [qtp2085002312-13] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:29:17.411368Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:29:17.411Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:29:17.411368Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:29:17.412Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/useraai HTTP/1.1 SC="500" 2022-04-26T06:29:17.418Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=83bc3a2d-2378-46bd-8e27-e18c64222e08 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:29:17.417861Z InvocationID=9e88c73a-5e64-44b2-8528-d8b40e638e86 LogTimestamp=2022-04-26T06:29:17.417949Z Entering 2022-04-26T06:29:17.418Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=83bc3a2d-2378-46bd-8e27-e18c64222e08 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:29:17.417861Z InvocationID=9e88c73a-5e64-44b2-8528-d8b40e638e86 LogTimestamp=2022-04-26T06:29:17.417949Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:29:17.423Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:29:17.418Z InvocationID=af2abb5f-7c43-4e82-b587-b6187cd8522a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id aai 2022-04-26T06:29:18.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ffc3f13-cb3a-46ca-b709-6bf3ae27c2a1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:29:18.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ffc3f13-cb3a-46ca-b709-6bf3ae27c2a1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@25d79e1d 2022-04-26T06:29:18.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ffc3f13-cb3a-46ca-b709-6bf3ae27c2a1 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:29:18.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ffc3f13-cb3a-46ca-b709-6bf3ae27c2a1 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:29:19.817Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=5ffc3f13-cb3a-46ca-b709-6bf3ae27c2a1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:29:19.818Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=5ffc3f13-cb3a-46ca-b709-6bf3ae27c2a1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:29:20.066Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:29:17.418Z InvocationID=af2abb5f-7c43-4e82-b587-b6187cd8522a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:29:20.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:29:20.385Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:29:22.139Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:29:17.418Z InvocationID=af2abb5f-7c43-4e82-b587-b6187cd8522a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:29:22.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=209f6075-2b70-4e07-90df-eae17d817a5f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:29:22.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=209f6075-2b70-4e07-90df-eae17d817a5f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7414fce8 2022-04-26T06:29:22.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=209f6075-2b70-4e07-90df-eae17d817a5f PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:29:22.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=209f6075-2b70-4e07-90df-eae17d817a5f PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:29:23.657Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=209f6075-2b70-4e07-90df-eae17d817a5f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:29:23.657Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=209f6075-2b70-4e07-90df-eae17d817a5f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:29:24.253Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:29:17.418Z InvocationID=af2abb5f-7c43-4e82-b587-b6187cd8522a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:29:25.452Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56626609-09a1-4cbf-8476-e82db90c1539 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:29:25.452Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56626609-09a1-4cbf-8476-e82db90c1539 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@716d134 2022-04-26T06:29:25.452Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56626609-09a1-4cbf-8476-e82db90c1539 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:29:25.452Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56626609-09a1-4cbf-8476-e82db90c1539 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:29:26.459Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:29:17.418Z InvocationID=af2abb5f-7c43-4e82-b587-b6187cd8522a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:29:26.472Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=56626609-09a1-4cbf-8476-e82db90c1539 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:29:26.473Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=56626609-09a1-4cbf-8476-e82db90c1539 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:29:28.436Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a832df9-efe9-4cb7-8bbd-7aca9221ea32 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:29:28.436Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a832df9-efe9-4cb7-8bbd-7aca9221ea32 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5567d530 2022-04-26T06:29:28.436Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a832df9-efe9-4cb7-8bbd-7aca9221ea32 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:29:28.436Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a832df9-efe9-4cb7-8bbd-7aca9221ea32 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:29:28.735Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:29:17.418Z InvocationID=af2abb5f-7c43-4e82-b587-b6187cd8522a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:29:29.459Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=7a832df9-efe9-4cb7-8bbd-7aca9221ea32 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:29:29.459Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=7a832df9-efe9-4cb7-8bbd-7aca9221ea32 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:29:31.142Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:29:17.418Z InvocationID=af2abb5f-7c43-4e82-b587-b6187cd8522a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:29:31.430Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bbd905d-77d8-4b6a-8495-88459629bc48 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:29:31.431Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bbd905d-77d8-4b6a-8495-88459629bc48 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3e2757f1 2022-04-26T06:29:31.431Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bbd905d-77d8-4b6a-8495-88459629bc48 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:29:31.431Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bbd905d-77d8-4b6a-8495-88459629bc48 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:29:32.462Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=2bbd905d-77d8-4b6a-8495-88459629bc48 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:29:32.463Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=2bbd905d-77d8-4b6a-8495-88459629bc48 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:29:34.029Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:29:17.418Z InvocationID=af2abb5f-7c43-4e82-b587-b6187cd8522a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:29:34.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=923a441c-7317-4a2d-8621-5691ec990103 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:29:34.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=923a441c-7317-4a2d-8621-5691ec990103 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6f2a2387 2022-04-26T06:29:34.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=923a441c-7317-4a2d-8621-5691ec990103 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:29:34.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=923a441c-7317-4a2d-8621-5691ec990103 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:29:35.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:29:35.377Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:29:35.471Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=923a441c-7317-4a2d-8621-5691ec990103 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:29:35.471Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=923a441c-7317-4a2d-8621-5691ec990103 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:29:36.058Z [cluster2-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:29:36.061Z [cluster2-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 30000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:29:37.442Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3c54e3d-2c50-48b9-8e2c-6ef7b3f7ddc4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:29:37.442Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3c54e3d-2c50-48b9-8e2c-6ef7b3f7ddc4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@29bfb7bd 2022-04-26T06:29:37.442Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3c54e3d-2c50-48b9-8e2c-6ef7b3f7ddc4 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:29:37.442Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3c54e3d-2c50-48b9-8e2c-6ef7b3f7ddc4 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:29:37.635Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:29:17.418Z InvocationID=af2abb5f-7c43-4e82-b587-b6187cd8522a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:29:38.477Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f3c54e3d-2c50-48b9-8e2c-6ef7b3f7ddc4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:29:38.478Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f3c54e3d-2c50-48b9-8e2c-6ef7b3f7ddc4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:29:39.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95a8fe5a-4208-4a94-9d9f-6b0f70ae632c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:29:39.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95a8fe5a-4208-4a94-9d9f-6b0f70ae632c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6e82f954 2022-04-26T06:29:39.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95a8fe5a-4208-4a94-9d9f-6b0f70ae632c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:29:39.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95a8fe5a-4208-4a94-9d9f-6b0f70ae632c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:29:40.813Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=95a8fe5a-4208-4a94-9d9f-6b0f70ae632c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:29:40.814Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=95a8fe5a-4208-4a94-9d9f-6b0f70ae632c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:29:41.243Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:29:17.418Z InvocationID=af2abb5f-7c43-4e82-b587-b6187cd8522a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:29:42.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2f58b51d-ef4b-46e1-919d-47dac8499539 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:29:42.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2f58b51d-ef4b-46e1-919d-47dac8499539 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8ec651b 2022-04-26T06:29:42.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2f58b51d-ef4b-46e1-919d-47dac8499539 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:29:42.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2f58b51d-ef4b-46e1-919d-47dac8499539 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:29:43.885Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=2f58b51d-ef4b-46e1-919d-47dac8499539 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:29:43.885Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=2f58b51d-ef4b-46e1-919d-47dac8499539 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:29:45.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7cfcc365-f3d2-4d98-bef6-f483a7fe3180 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:29:45.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7cfcc365-f3d2-4d98-bef6-f483a7fe3180 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@77279e06 2022-04-26T06:29:45.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7cfcc365-f3d2-4d98-bef6-f483a7fe3180 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:29:45.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7cfcc365-f3d2-4d98-bef6-f483a7fe3180 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:29:46.448Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:29:17.418Z InvocationID=af2abb5f-7c43-4e82-b587-b6187cd8522a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:29:46.793Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=7cfcc365-f3d2-4d98-bef6-f483a7fe3180 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:29:46.794Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=7cfcc365-f3d2-4d98-bef6-f483a7fe3180 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:29:48.792Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f116ffb-76d0-4368-9896-cf48f65055ed PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:29:48.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f116ffb-76d0-4368-9896-cf48f65055ed PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@11a41428 2022-04-26T06:29:48.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f116ffb-76d0-4368-9896-cf48f65055ed PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:29:48.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f116ffb-76d0-4368-9896-cf48f65055ed PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:29:49.833Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=4f116ffb-76d0-4368-9896-cf48f65055ed ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:29:49.833Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=4f116ffb-76d0-4368-9896-cf48f65055ed ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:29:50.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:29:50.378Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:29:52.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e6eb154-c30c-4367-aa45-1938f0e519b6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:29:52.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e6eb154-c30c-4367-aa45-1938f0e519b6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@60e2dc54 2022-04-26T06:29:52.639Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e6eb154-c30c-4367-aa45-1938f0e519b6 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:29:52.639Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e6eb154-c30c-4367-aa45-1938f0e519b6 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:29:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:29:52.686Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:29:52.686Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:29:53.673Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=4e6eb154-c30c-4367-aa45-1938f0e519b6 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:29:53.673Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=4e6eb154-c30c-4367-aa45-1938f0e519b6 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:29:55.143Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:29:17.418Z InvocationID=af2abb5f-7c43-4e82-b587-b6187cd8522a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteFailureException: Cassandra failure during write query at consistency ALL (3 responses were required but only 2 replica responded, 1 failed) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteFailureException: Cassandra failure during write query at consistency ALL (3 responses were required but only 2 replica responded, 1 failed) at com.datastax.driver.core.exceptions.WriteFailureException.copy(WriteFailureException.java:186) at com.datastax.driver.core.Responses$Error.asException(Responses.java:171) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteFailureException: Cassandra failure during write query at consistency ALL (3 responses were required but only 2 replica responded, 1 failed) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:112) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:29:55.834Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=030e1680-176e-4e40-a1da-b4551928dcd4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:29:55.834Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=030e1680-176e-4e40-a1da-b4551928dcd4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@706a0e4a 2022-04-26T06:29:55.834Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=030e1680-176e-4e40-a1da-b4551928dcd4 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:29:55.835Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=030e1680-176e-4e40-a1da-b4551928dcd4 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:29:56.873Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=030e1680-176e-4e40-a1da-b4551928dcd4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:29:56.874Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=030e1680-176e-4e40-a1da-b4551928dcd4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:29:58.539Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d261bbb-0aaa-4649-a3b0-5813deaade3c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:29:58.539Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d261bbb-0aaa-4649-a3b0-5813deaade3c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5838ab8d 2022-04-26T06:29:58.539Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d261bbb-0aaa-4649-a3b0-5813deaade3c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:29:58.540Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d261bbb-0aaa-4649-a3b0-5813deaade3c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:29:59.561Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=9d261bbb-0aaa-4649-a3b0-5813deaade3c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:29:59.562Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=9d261bbb-0aaa-4649-a3b0-5813deaade3c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:30:01.344Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5708905a-36e1-4480-8af3-86c5acbd16f3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:30:01.344Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5708905a-36e1-4480-8af3-86c5acbd16f3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4bd52241 2022-04-26T06:30:01.344Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5708905a-36e1-4480-8af3-86c5acbd16f3 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:30:01.344Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5708905a-36e1-4480-8af3-86c5acbd16f3 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:30:02.381Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=5708905a-36e1-4480-8af3-86c5acbd16f3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:30:02.381Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=5708905a-36e1-4480-8af3-86c5acbd16f3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:30:03.548Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:29:17.418Z InvocationID=af2abb5f-7c43-4e82-b587-b6187cd8522a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:30:04.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c014ae6c-4ae7-4bf6-a25d-c36633f3225d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:30:04.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c014ae6c-4ae7-4bf6-a25d-c36633f3225d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@e2961d1 2022-04-26T06:30:04.435Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c014ae6c-4ae7-4bf6-a25d-c36633f3225d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:30:04.435Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c014ae6c-4ae7-4bf6-a25d-c36633f3225d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:30:05.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:30:05.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:30:05.453Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c014ae6c-4ae7-4bf6-a25d-c36633f3225d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:30:05.453Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c014ae6c-4ae7-4bf6-a25d-c36633f3225d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:30:07.337Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=07109c07-2255-4d9a-a2ea-03864b4b412c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:30:07.337Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=07109c07-2255-4d9a-a2ea-03864b4b412c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6c28813f 2022-04-26T06:30:07.338Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=07109c07-2255-4d9a-a2ea-03864b4b412c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:30:07.338Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=07109c07-2255-4d9a-a2ea-03864b4b412c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:30:07.652Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:30:08.361Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=07109c07-2255-4d9a-a2ea-03864b4b412c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:30:08.361Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=07109c07-2255-4d9a-a2ea-03864b4b412c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:30:09.066Z [cluster1-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:30:09.066Z [cluster2-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:30:09.069Z [cluster1-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 256000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:30:09.069Z [cluster2-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 30000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:30:09.788Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e556078-e995-4592-a5df-a09ecef83dbb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:30:09.788Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e556078-e995-4592-a5df-a09ecef83dbb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3c28c632 2022-04-26T06:30:09.788Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e556078-e995-4592-a5df-a09ecef83dbb PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:30:09.788Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e556078-e995-4592-a5df-a09ecef83dbb PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:30:10.825Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=4e556078-e995-4592-a5df-a09ecef83dbb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:30:10.826Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=4e556078-e995-4592-a5df-a09ecef83dbb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:30:12.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9284e113-9e35-4243-b39d-677b6da9bff5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:30:12.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9284e113-9e35-4243-b39d-677b6da9bff5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@71a53684 2022-04-26T06:30:12.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9284e113-9e35-4243-b39d-677b6da9bff5 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:30:12.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9284e113-9e35-4243-b39d-677b6da9bff5 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:30:13.800Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=9284e113-9e35-4243-b39d-677b6da9bff5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:30:13.801Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=9284e113-9e35-4243-b39d-677b6da9bff5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:30:15.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d28559cc-31d4-4641-9fa9-c63cf0520977 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:30:15.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d28559cc-31d4-4641-9fa9-c63cf0520977 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@221c9067 2022-04-26T06:30:15.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d28559cc-31d4-4641-9fa9-c63cf0520977 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:30:15.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d28559cc-31d4-4641-9fa9-c63cf0520977 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:30:16.813Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=d28559cc-31d4-4641-9fa9-c63cf0520977 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:30:16.814Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=d28559cc-31d4-4641-9fa9-c63cf0520977 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:30:18.353Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:29:17.418Z InvocationID=af2abb5f-7c43-4e82-b587-b6187cd8522a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:30:18.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c5516e4b-5b43-45a1-8a49-5b06d35beaa8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:30:18.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c5516e4b-5b43-45a1-8a49-5b06d35beaa8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@40ecce08 2022-04-26T06:30:18.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c5516e4b-5b43-45a1-8a49-5b06d35beaa8 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:30:18.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c5516e4b-5b43-45a1-8a49-5b06d35beaa8 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:30:19.817Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c5516e4b-5b43-45a1-8a49-5b06d35beaa8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:30:19.818Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c5516e4b-5b43-45a1-8a49-5b06d35beaa8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:30:20.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:30:20.374Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:30:22.740Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b4a4152-fd44-443c-b969-5b27ca3f974a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:30:22.741Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b4a4152-fd44-443c-b969-5b27ca3f974a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@20cc86b0 2022-04-26T06:30:22.741Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b4a4152-fd44-443c-b969-5b27ca3f974a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:30:22.741Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b4a4152-fd44-443c-b969-5b27ca3f974a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:30:23.757Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=5b4a4152-fd44-443c-b969-5b27ca3f974a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:30:23.757Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=5b4a4152-fd44-443c-b969-5b27ca3f974a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:30:25.440Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b65a5ca-c031-4cfa-a1eb-aa8821bcb58e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:30:25.441Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b65a5ca-c031-4cfa-a1eb-aa8821bcb58e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@30b1d95a 2022-04-26T06:30:25.441Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b65a5ca-c031-4cfa-a1eb-aa8821bcb58e PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:30:25.441Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b65a5ca-c031-4cfa-a1eb-aa8821bcb58e PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:30:25.856Z [JanusGraph Cluster-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:30:25.857Z [JanusGraph Cluster-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 256000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:30:26.476Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=4b65a5ca-c031-4cfa-a1eb-aa8821bcb58e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:30:26.477Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=4b65a5ca-c031-4cfa-a1eb-aa8821bcb58e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:30:28.544Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=080ce0e1-7967-4e0b-a0ee-075c8c0935c2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:30:28.544Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=080ce0e1-7967-4e0b-a0ee-075c8c0935c2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3f3f92e2 2022-04-26T06:30:28.545Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=080ce0e1-7967-4e0b-a0ee-075c8c0935c2 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:30:28.545Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=080ce0e1-7967-4e0b-a0ee-075c8c0935c2 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:30:29.577Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=080ce0e1-7967-4e0b-a0ee-075c8c0935c2 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:30:29.577Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=080ce0e1-7967-4e0b-a0ee-075c8c0935c2 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:30:31.541Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6c73381-e87a-4799-aa6e-afcd2dba976a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:30:31.542Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6c73381-e87a-4799-aa6e-afcd2dba976a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@49710b1c 2022-04-26T06:30:31.542Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6c73381-e87a-4799-aa6e-afcd2dba976a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:30:31.542Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6c73381-e87a-4799-aa6e-afcd2dba976a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:30:32.554Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f6c73381-e87a-4799-aa6e-afcd2dba976a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:30:32.555Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f6c73381-e87a-4799-aa6e-afcd2dba976a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:30:34.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=051f28bf-053f-416f-ac70-9205c866bf86 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:30:34.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=051f28bf-053f-416f-ac70-9205c866bf86 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@62617d95 2022-04-26T06:30:34.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=051f28bf-053f-416f-ac70-9205c866bf86 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:30:34.531Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=051f28bf-053f-416f-ac70-9205c866bf86 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:30:35.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:30:35.384Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:30:35.562Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=051f28bf-053f-416f-ac70-9205c866bf86 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:30:35.562Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=051f28bf-053f-416f-ac70-9205c866bf86 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:30:37.333Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9f2976d-8fa3-4257-aeaa-e0062a7acf61 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:30:37.334Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9f2976d-8fa3-4257-aeaa-e0062a7acf61 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@775493ef 2022-04-26T06:30:37.334Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9f2976d-8fa3-4257-aeaa-e0062a7acf61 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:30:37.334Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9f2976d-8fa3-4257-aeaa-e0062a7acf61 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:30:38.377Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f9f2976d-8fa3-4257-aeaa-e0062a7acf61 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:30:38.377Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f9f2976d-8fa3-4257-aeaa-e0062a7acf61 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:30:39.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=efe2f002-d4bb-45f3-9381-c485ecb9f998 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:30:39.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=efe2f002-d4bb-45f3-9381-c485ecb9f998 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6d5722e4 2022-04-26T06:30:39.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=efe2f002-d4bb-45f3-9381-c485ecb9f998 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:30:39.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=efe2f002-d4bb-45f3-9381-c485ecb9f998 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:30:40.812Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=efe2f002-d4bb-45f3-9381-c485ecb9f998 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:30:40.813Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=efe2f002-d4bb-45f3-9381-c485ecb9f998 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:30:42.787Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=49cc1615-ca27-41b5-a590-8d4033a80f31 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:30:42.787Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=49cc1615-ca27-41b5-a590-8d4033a80f31 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6a19359a 2022-04-26T06:30:42.787Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=49cc1615-ca27-41b5-a590-8d4033a80f31 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:30:42.787Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=49cc1615-ca27-41b5-a590-8d4033a80f31 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:30:43.821Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=49cc1615-ca27-41b5-a590-8d4033a80f31 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:30:43.821Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=49cc1615-ca27-41b5-a590-8d4033a80f31 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:30:45.056Z [cluster2-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:30:45.057Z [cluster2-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 30000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:30:45.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd094523-7dd0-4fb5-9c0e-2ae2e3f79c67 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:30:45.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd094523-7dd0-4fb5-9c0e-2ae2e3f79c67 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2170ec5b 2022-04-26T06:30:45.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd094523-7dd0-4fb5-9c0e-2ae2e3f79c67 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:30:45.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd094523-7dd0-4fb5-9c0e-2ae2e3f79c67 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:30:45.959Z [qtp2085002312-16] ERROR o.j.g.database.StandardJanusGraph RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:29:17.418Z InvocationID=af2abb5f-7c43-4e82-b587-b6187cd8522a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Could not commit transaction [353] due to storage exception in commit org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT1M40S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 104 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 105 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:30:45.960Z [qtp2085002312-16] ERROR o.j.g.database.StandardJanusGraph RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:29:17.418Z InvocationID=af2abb5f-7c43-4e82-b587-b6187cd8522a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Could not commit transaction [353] due to exception org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT1M40S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 104 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 105 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:30:46.500Z [qtp2085002312-16] WARN o.o.sdc.be.servlets.ConsumerServlet RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:29:17.418Z InvocationID=af2abb5f-7c43-4e82-b587-b6187cd8522a PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb #createConsumer('{"consumerName": "aai", "consumerSalt": "9cd4c3ad2a6f6ce3f3414e68b5157e63", "consumerPassword": "35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495"}', 'Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/consumers)@6d0411d6', 'jh0003'): org.apache.cxf.jaxrs.impl.ResponseImpl@7b6b4f63 in 1.48min (too slow!) 2022-04-26T06:30:46.501Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=89083 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:29:17.418Z InvocationID=af2abb5f-7c43-4e82-b587-b6187cd8522a LogTimestamp=2022-04-26T06:30:46.501390Z ResponseDesc=Created Exiting. 2022-04-26T06:30:46.502Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:30:46.515Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=28641 PartnerName=kube-probe/1.22 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=ERROR ServerIPAddress=10.233.65.41 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2022-04-26T06:30:17.874Z InvocationID=534d61d2-d7c3-4296-a558-636131f24972 ErrorCode=500 LogTimestamp=2022-04-26T06:30:46.515344Z ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:30:46.516Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userappc HTTP/1.1 SC="500" 2022-04-26T06:30:46.519Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=9b83107c-444a-4223-a71a-dbf3d70c51f2 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2022-04-26T06:30:46.519628Z InvocationID=c4e167e4-7130-4595-8c31-676b98d0f4aa LogTimestamp=2022-04-26T06:30:46.519658Z Entering 2022-04-26T06:30:46.520Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=9b83107c-444a-4223-a71a-dbf3d70c51f2 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2022-04-26T06:30:46.519628Z InvocationID=c4e167e4-7130-4595-8c31-676b98d0f4aa LogTimestamp=2022-04-26T06:30:46.519658Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:30:46.522Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:30:46.520Z InvocationID=c24778bd-af5f-436c-bec3-6ed422b5c8da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id appc 2022-04-26T06:30:46.798Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=fd094523-7dd0-4fb5-9c0e-2ae2e3f79c67 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:30:46.799Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=fd094523-7dd0-4fb5-9c0e-2ae2e3f79c67 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:30:48.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=74c38e47-78e4-4448-98bf-30e3bf5b58a9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:30:48.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=74c38e47-78e4-4448-98bf-30e3bf5b58a9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@33c34cdc 2022-04-26T06:30:48.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=74c38e47-78e4-4448-98bf-30e3bf5b58a9 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:30:48.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=74c38e47-78e4-4448-98bf-30e3bf5b58a9 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:30:49.044Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:30:46.520Z InvocationID=c24778bd-af5f-436c-bec3-6ed422b5c8da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:30:49.801Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=74c38e47-78e4-4448-98bf-30e3bf5b58a9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:30:49.801Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=74c38e47-78e4-4448-98bf-30e3bf5b58a9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:30:50.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:30:50.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:30:51.074Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:30:46.520Z InvocationID=c24778bd-af5f-436c-bec3-6ed422b5c8da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:30:52.637Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a1ff5991-7607-4faf-b782-3340b46a439b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:30:52.637Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a1ff5991-7607-4faf-b782-3340b46a439b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@45432c0c 2022-04-26T06:30:52.637Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a1ff5991-7607-4faf-b782-3340b46a439b PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:30:52.637Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a1ff5991-7607-4faf-b782-3340b46a439b PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:30:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:30:52.686Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:30:52.686Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:30:53.134Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:30:46.520Z InvocationID=c24778bd-af5f-436c-bec3-6ed422b5c8da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:30:53.673Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=a1ff5991-7607-4faf-b782-3340b46a439b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:30:53.673Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=a1ff5991-7607-4faf-b782-3340b46a439b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:30:55.239Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:30:46.520Z InvocationID=c24778bd-af5f-436c-bec3-6ed422b5c8da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:30:55.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04e7d7c8-3db8-48c6-8677-ebb787c76f1c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:30:55.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04e7d7c8-3db8-48c6-8677-ebb787c76f1c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@54d7c480 2022-04-26T06:30:55.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04e7d7c8-3db8-48c6-8677-ebb787c76f1c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:30:55.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04e7d7c8-3db8-48c6-8677-ebb787c76f1c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:30:56.460Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=04e7d7c8-3db8-48c6-8677-ebb787c76f1c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:30:56.461Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=04e7d7c8-3db8-48c6-8677-ebb787c76f1c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:30:57.345Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:30:46.520Z InvocationID=c24778bd-af5f-436c-bec3-6ed422b5c8da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:30:58.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1b45cfa-b75d-493c-96de-c9226ea06a73 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:30:58.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1b45cfa-b75d-493c-96de-c9226ea06a73 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@533499d0 2022-04-26T06:30:58.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1b45cfa-b75d-493c-96de-c9226ea06a73 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:30:58.435Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1b45cfa-b75d-493c-96de-c9226ea06a73 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:30:59.465Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f1b45cfa-b75d-493c-96de-c9226ea06a73 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:30:59.466Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f1b45cfa-b75d-493c-96de-c9226ea06a73 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:30:59.531Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:30:46.520Z InvocationID=c24778bd-af5f-436c-bec3-6ed422b5c8da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:31:01.340Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34c196b1-b989-4e5a-b666-6a98520f8ca7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:31:01.341Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34c196b1-b989-4e5a-b666-6a98520f8ca7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1a4769b8 2022-04-26T06:31:01.341Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34c196b1-b989-4e5a-b666-6a98520f8ca7 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:31:01.341Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34c196b1-b989-4e5a-b666-6a98520f8ca7 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:31:01.738Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:30:46.520Z InvocationID=c24778bd-af5f-436c-bec3-6ed422b5c8da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:31:02.377Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=34c196b1-b989-4e5a-b666-6a98520f8ca7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:31:02.377Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=34c196b1-b989-4e5a-b666-6a98520f8ca7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:31:04.151Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:30:46.520Z InvocationID=c24778bd-af5f-436c-bec3-6ed422b5c8da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:31:04.340Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95ae41a9-fbfb-4305-9895-92d84fde86e6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:31:04.340Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95ae41a9-fbfb-4305-9895-92d84fde86e6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3dea4f23 2022-04-26T06:31:04.340Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95ae41a9-fbfb-4305-9895-92d84fde86e6 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:31:04.340Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95ae41a9-fbfb-4305-9895-92d84fde86e6 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:31:05.353Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=95ae41a9-fbfb-4305-9895-92d84fde86e6 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:31:05.353Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=95ae41a9-fbfb-4305-9895-92d84fde86e6 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:31:05.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:31:05.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:31:06.629Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:30:46.520Z InvocationID=c24778bd-af5f-436c-bec3-6ed422b5c8da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:31:07.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4caf8a9e-a4b4-4751-919b-3e62e18f8850 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:31:07.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4caf8a9e-a4b4-4751-919b-3e62e18f8850 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1e1acde6 2022-04-26T06:31:07.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4caf8a9e-a4b4-4751-919b-3e62e18f8850 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:31:07.430Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4caf8a9e-a4b4-4751-919b-3e62e18f8850 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:31:07.724Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:31:08.457Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=4caf8a9e-a4b4-4751-919b-3e62e18f8850 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:31:08.457Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=4caf8a9e-a4b4-4751-919b-3e62e18f8850 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:31:09.041Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:30:46.520Z InvocationID=c24778bd-af5f-436c-bec3-6ed422b5c8da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:31:09.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9ff3f03-f4b7-4a19-9dbf-16e2a3763072 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:31:09.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9ff3f03-f4b7-4a19-9dbf-16e2a3763072 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6dbd8c6 2022-04-26T06:31:09.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9ff3f03-f4b7-4a19-9dbf-16e2a3763072 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:31:09.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9ff3f03-f4b7-4a19-9dbf-16e2a3763072 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:31:10.793Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f9ff3f03-f4b7-4a19-9dbf-16e2a3763072 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:31:10.793Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f9ff3f03-f4b7-4a19-9dbf-16e2a3763072 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:31:11.445Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:30:46.520Z InvocationID=c24778bd-af5f-436c-bec3-6ed422b5c8da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:31:12.833Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2313fcf9-2022-4e46-b775-576f71e963a2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:31:12.833Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2313fcf9-2022-4e46-b775-576f71e963a2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7aabdeb4 2022-04-26T06:31:12.833Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2313fcf9-2022-4e46-b775-576f71e963a2 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:31:12.833Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2313fcf9-2022-4e46-b775-576f71e963a2 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:31:13.865Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=2313fcf9-2022-4e46-b775-576f71e963a2 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:31:13.865Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=2313fcf9-2022-4e46-b775-576f71e963a2 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:31:14.250Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:30:46.520Z InvocationID=c24778bd-af5f-436c-bec3-6ed422b5c8da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:31:15.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=305f7cbd-a97a-465b-b7e8-bfeca6bf552d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:31:15.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=305f7cbd-a97a-465b-b7e8-bfeca6bf552d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@73942b12 2022-04-26T06:31:15.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=305f7cbd-a97a-465b-b7e8-bfeca6bf552d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:31:15.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=305f7cbd-a97a-465b-b7e8-bfeca6bf552d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:31:16.809Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=305f7cbd-a97a-465b-b7e8-bfeca6bf552d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:31:16.809Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=305f7cbd-a97a-465b-b7e8-bfeca6bf552d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:31:17.855Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:30:46.520Z InvocationID=c24778bd-af5f-436c-bec3-6ed422b5c8da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:31:18.785Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08e0bc68-31e3-44a2-b5e1-8358ea8ab2b6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:31:18.785Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08e0bc68-31e3-44a2-b5e1-8358ea8ab2b6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@577fb20f 2022-04-26T06:31:18.785Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08e0bc68-31e3-44a2-b5e1-8358ea8ab2b6 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:31:18.786Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08e0bc68-31e3-44a2-b5e1-8358ea8ab2b6 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:31:19.817Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=08e0bc68-31e3-44a2-b5e1-8358ea8ab2b6 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:31:19.818Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=08e0bc68-31e3-44a2-b5e1-8358ea8ab2b6 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:31:20.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:31:20.379Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:31:21.461Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:30:46.520Z InvocationID=c24778bd-af5f-436c-bec3-6ed422b5c8da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:31:22.362Z [cluster2-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:31:22.363Z [cluster2-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 30000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:31:22.637Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c59c51cb-97d6-48f0-b1d1-e5ca1141ef3d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:31:22.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c59c51cb-97d6-48f0-b1d1-e5ca1141ef3d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@204fb1ae 2022-04-26T06:31:22.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c59c51cb-97d6-48f0-b1d1-e5ca1141ef3d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:31:22.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c59c51cb-97d6-48f0-b1d1-e5ca1141ef3d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:31:23.658Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c59c51cb-97d6-48f0-b1d1-e5ca1141ef3d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:31:23.658Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c59c51cb-97d6-48f0-b1d1-e5ca1141ef3d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:31:25.435Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76ee8767-8801-4b7b-a9ea-48ce05ba5356 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:31:25.435Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76ee8767-8801-4b7b-a9ea-48ce05ba5356 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@55b5cb0d 2022-04-26T06:31:25.435Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76ee8767-8801-4b7b-a9ea-48ce05ba5356 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:31:25.435Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76ee8767-8801-4b7b-a9ea-48ce05ba5356 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:31:26.477Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=76ee8767-8801-4b7b-a9ea-48ce05ba5356 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:31:26.478Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=76ee8767-8801-4b7b-a9ea-48ce05ba5356 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:31:26.735Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:30:46.520Z InvocationID=c24778bd-af5f-436c-bec3-6ed422b5c8da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:31:28.435Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46c6258b-2a46-4e9c-8059-f12a25717b4d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:31:28.435Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46c6258b-2a46-4e9c-8059-f12a25717b4d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@74c5e482 2022-04-26T06:31:28.435Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46c6258b-2a46-4e9c-8059-f12a25717b4d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:31:28.435Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46c6258b-2a46-4e9c-8059-f12a25717b4d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:31:29.449Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=46c6258b-2a46-4e9c-8059-f12a25717b4d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:31:29.450Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=46c6258b-2a46-4e9c-8059-f12a25717b4d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:31:31.332Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41547ce1-9b1b-4383-9223-0709c60d021e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:31:31.332Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41547ce1-9b1b-4383-9223-0709c60d021e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@77d29ddb 2022-04-26T06:31:31.333Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41547ce1-9b1b-4383-9223-0709c60d021e PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:31:31.333Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41547ce1-9b1b-4383-9223-0709c60d021e PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:31:31.944Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:30:46.520Z InvocationID=c24778bd-af5f-436c-bec3-6ed422b5c8da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:31:32.365Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=41547ce1-9b1b-4383-9223-0709c60d021e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:31:32.366Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=41547ce1-9b1b-4383-9223-0709c60d021e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:31:34.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=507918c7-5422-4a21-8d58-9477674054f7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:31:34.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=507918c7-5422-4a21-8d58-9477674054f7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@525a89ff 2022-04-26T06:31:34.430Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=507918c7-5422-4a21-8d58-9477674054f7 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:31:34.430Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=507918c7-5422-4a21-8d58-9477674054f7 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:31:35.368Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:31:35.378Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:31:35.469Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=507918c7-5422-4a21-8d58-9477674054f7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:31:35.469Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=507918c7-5422-4a21-8d58-9477674054f7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:31:37.340Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=101e6188-c211-48bb-a0df-1cb453f13118 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:31:37.341Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=101e6188-c211-48bb-a0df-1cb453f13118 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1b7c1cf9 2022-04-26T06:31:37.341Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=101e6188-c211-48bb-a0df-1cb453f13118 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:31:37.341Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=101e6188-c211-48bb-a0df-1cb453f13118 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:31:38.377Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=101e6188-c211-48bb-a0df-1cb453f13118 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:31:38.378Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=101e6188-c211-48bb-a0df-1cb453f13118 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:31:39.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40f3e687-d5fd-46a6-81af-977999d7a724 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:31:39.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40f3e687-d5fd-46a6-81af-977999d7a724 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@10ef4157 2022-04-26T06:31:39.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40f3e687-d5fd-46a6-81af-977999d7a724 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:31:39.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40f3e687-d5fd-46a6-81af-977999d7a724 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:31:40.350Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:30:46.520Z InvocationID=c24778bd-af5f-436c-bec3-6ed422b5c8da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:31:40.816Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=40f3e687-d5fd-46a6-81af-977999d7a724 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:31:40.816Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=40f3e687-d5fd-46a6-81af-977999d7a724 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:31:42.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ee9d29a-d364-461b-9fac-a47ebb7c1534 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:31:42.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ee9d29a-d364-461b-9fac-a47ebb7c1534 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@89c0089 2022-04-26T06:31:42.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ee9d29a-d364-461b-9fac-a47ebb7c1534 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:31:42.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ee9d29a-d364-461b-9fac-a47ebb7c1534 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:31:43.885Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3ee9d29a-d364-461b-9fac-a47ebb7c1534 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:31:43.885Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3ee9d29a-d364-461b-9fac-a47ebb7c1534 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:31:45.789Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c0b3ff1-feac-4a8a-9192-fab42d665de5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:31:45.789Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c0b3ff1-feac-4a8a-9192-fab42d665de5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@15a05140 2022-04-26T06:31:45.789Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c0b3ff1-feac-4a8a-9192-fab42d665de5 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:31:45.789Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c0b3ff1-feac-4a8a-9192-fab42d665de5 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:31:46.828Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=8c0b3ff1-feac-4a8a-9192-fab42d665de5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:31:46.828Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=8c0b3ff1-feac-4a8a-9192-fab42d665de5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:31:48.847Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0efe352a-7c08-4745-9d98-59768d9a0994 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:31:48.847Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0efe352a-7c08-4745-9d98-59768d9a0994 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6c33f86f 2022-04-26T06:31:48.847Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0efe352a-7c08-4745-9d98-59768d9a0994 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:31:48.848Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0efe352a-7c08-4745-9d98-59768d9a0994 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:31:49.897Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=0efe352a-7c08-4745-9d98-59768d9a0994 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:31:49.897Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=0efe352a-7c08-4745-9d98-59768d9a0994 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:31:50.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:31:50.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:31:52.536Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f320e7db-9a9d-4bbd-8732-a4105953bdab PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:31:52.536Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f320e7db-9a9d-4bbd-8732-a4105953bdab PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e34a6c8 2022-04-26T06:31:52.536Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f320e7db-9a9d-4bbd-8732-a4105953bdab PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:31:52.536Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f320e7db-9a9d-4bbd-8732-a4105953bdab PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:31:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:31:52.687Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:31:52.687Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:31:53.577Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f320e7db-9a9d-4bbd-8732-a4105953bdab ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:31:53.577Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f320e7db-9a9d-4bbd-8732-a4105953bdab ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:31:55.245Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:30:46.520Z InvocationID=c24778bd-af5f-436c-bec3-6ed422b5c8da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:31:55.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e2658dc-5108-4a12-8162-96fc56cb7ba7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:31:55.440Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e2658dc-5108-4a12-8162-96fc56cb7ba7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@10284faa 2022-04-26T06:31:55.440Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e2658dc-5108-4a12-8162-96fc56cb7ba7 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:31:55.440Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e2658dc-5108-4a12-8162-96fc56cb7ba7 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:31:56.457Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=0e2658dc-5108-4a12-8162-96fc56cb7ba7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:31:56.457Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=0e2658dc-5108-4a12-8162-96fc56cb7ba7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:31:58.267Z [cluster2-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:31:58.268Z [cluster2-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 30000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:31:58.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6bbaa4fb-e2b7-4904-acea-477f80b4003a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:31:58.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6bbaa4fb-e2b7-4904-acea-477f80b4003a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6c583199 2022-04-26T06:31:58.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6bbaa4fb-e2b7-4904-acea-477f80b4003a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:31:58.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6bbaa4fb-e2b7-4904-acea-477f80b4003a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:31:59.465Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=6bbaa4fb-e2b7-4904-acea-477f80b4003a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:31:59.470Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=6bbaa4fb-e2b7-4904-acea-477f80b4003a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:32:01.430Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a6fe2d8-645a-4c2c-81d0-22ba667d0ad4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:32:01.430Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a6fe2d8-645a-4c2c-81d0-22ba667d0ad4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4d77d7e 2022-04-26T06:32:01.431Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a6fe2d8-645a-4c2c-81d0-22ba667d0ad4 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:32:01.431Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a6fe2d8-645a-4c2c-81d0-22ba667d0ad4 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:32:02.443Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=2a6fe2d8-645a-4c2c-81d0-22ba667d0ad4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:32:02.444Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=2a6fe2d8-645a-4c2c-81d0-22ba667d0ad4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:32:04.431Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0bd47ad-4d19-4611-a337-30b69ad3e06a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:32:04.431Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0bd47ad-4d19-4611-a337-30b69ad3e06a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@100ca796 2022-04-26T06:32:04.431Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0bd47ad-4d19-4611-a337-30b69ad3e06a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:32:04.431Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0bd47ad-4d19-4611-a337-30b69ad3e06a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:32:05.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:32:05.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:32:05.454Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=a0bd47ad-4d19-4611-a337-30b69ad3e06a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:32:05.454Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=a0bd47ad-4d19-4611-a337-30b69ad3e06a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:32:07.428Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ead2cef5-c501-44d6-88c4-7016b077bb10 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:32:07.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ead2cef5-c501-44d6-88c4-7016b077bb10 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7860813c 2022-04-26T06:32:07.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ead2cef5-c501-44d6-88c4-7016b077bb10 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:32:07.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ead2cef5-c501-44d6-88c4-7016b077bb10 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:32:07.689Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:32:08.466Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ead2cef5-c501-44d6-88c4-7016b077bb10 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:32:08.467Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ead2cef5-c501-44d6-88c4-7016b077bb10 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:32:09.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9b03573b-a406-4582-907f-edf7e89736d0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:32:09.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9b03573b-a406-4582-907f-edf7e89736d0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1c2be619 2022-04-26T06:32:09.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9b03573b-a406-4582-907f-edf7e89736d0 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:32:09.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9b03573b-a406-4582-907f-edf7e89736d0 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:32:10.051Z [qtp2085002312-13] ERROR o.j.g.database.StandardJanusGraph RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:30:46.520Z InvocationID=c24778bd-af5f-436c-bec3-6ed422b5c8da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Could not commit transaction [356] due to storage exception in commit org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT1M40S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 103 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 104 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:32:10.052Z [qtp2085002312-13] ERROR o.j.g.database.StandardJanusGraph RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:30:46.520Z InvocationID=c24778bd-af5f-436c-bec3-6ed422b5c8da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Could not commit transaction [356] due to exception org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT1M40S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 103 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 104 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:32:10.596Z [qtp2085002312-13] WARN o.o.sdc.be.servlets.ConsumerServlet RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:30:46.520Z InvocationID=c24778bd-af5f-436c-bec3-6ed422b5c8da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb #createConsumer('{"consumerName": "appc", "consumerSalt": "9cd4c3ad2a6f6ce3f3414e68b5157e63", "consumerPassword": "35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495"}', 'Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/consumers)@6d0411d6', 'jh0003'): org.apache.cxf.jaxrs.impl.ResponseImpl@76e44662 in 1.40min (too slow!) 2022-04-26T06:32:10.599Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=84078 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:30:46.520Z InvocationID=c24778bd-af5f-436c-bec3-6ed422b5c8da LogTimestamp=2022-04-26T06:32:10.598064Z ResponseDesc=Created Exiting. 2022-04-26T06:32:10.599Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:32:10.612Z [qtp2085002312-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:32:10.612136Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:32:10.612Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:32:10.612136Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:32:10.613Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdcae HTTP/1.1 SC="500" 2022-04-26T06:32:10.629Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0a739309-c626-4295-9b68-dc1c10c34761 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:32:10.629124Z InvocationID=b5ad71ea-7b15-40fb-9ade-3fb24f4374a8 LogTimestamp=2022-04-26T06:32:10.629176Z Entering 2022-04-26T06:32:10.629Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0a739309-c626-4295-9b68-dc1c10c34761 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:32:10.629124Z InvocationID=b5ad71ea-7b15-40fb-9ade-3fb24f4374a8 LogTimestamp=2022-04-26T06:32:10.629176Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:32:10.633Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:32:10.630Z InvocationID=f4440787-6ba0-4b1a-9e12-001972981ba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id dcae 2022-04-26T06:32:10.798Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=9b03573b-a406-4582-907f-edf7e89736d0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:32:10.799Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=9b03573b-a406-4582-907f-edf7e89736d0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:32:12.837Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1b5a247-8f04-45ad-8bb8-dbd6eb0486c5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:32:12.837Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1b5a247-8f04-45ad-8bb8-dbd6eb0486c5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3e42e708 2022-04-26T06:32:12.837Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1b5a247-8f04-45ad-8bb8-dbd6eb0486c5 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:32:12.838Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1b5a247-8f04-45ad-8bb8-dbd6eb0486c5 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:32:13.156Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:32:10.630Z InvocationID=f4440787-6ba0-4b1a-9e12-001972981ba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:32:13.865Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e1b5a247-8f04-45ad-8bb8-dbd6eb0486c5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:32:13.866Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e1b5a247-8f04-45ad-8bb8-dbd6eb0486c5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:32:15.212Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:32:10.630Z InvocationID=f4440787-6ba0-4b1a-9e12-001972981ba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:32:15.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a01d26c8-9b38-4c19-9db6-0828ce64478f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:32:15.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a01d26c8-9b38-4c19-9db6-0828ce64478f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@628e3e1c 2022-04-26T06:32:15.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a01d26c8-9b38-4c19-9db6-0828ce64478f PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:32:15.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a01d26c8-9b38-4c19-9db6-0828ce64478f PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:32:16.809Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=a01d26c8-9b38-4c19-9db6-0828ce64478f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:32:16.809Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=a01d26c8-9b38-4c19-9db6-0828ce64478f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:32:17.317Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:32:10.630Z InvocationID=f4440787-6ba0-4b1a-9e12-001972981ba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:32:18.841Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a30f74a3-eb5e-4cfd-96d7-46154c6b2872 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:32:18.841Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a30f74a3-eb5e-4cfd-96d7-46154c6b2872 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@364f8c01 2022-04-26T06:32:18.841Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a30f74a3-eb5e-4cfd-96d7-46154c6b2872 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:32:18.841Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a30f74a3-eb5e-4cfd-96d7-46154c6b2872 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:32:19.523Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:32:10.630Z InvocationID=f4440787-6ba0-4b1a-9e12-001972981ba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:32:19.881Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=a30f74a3-eb5e-4cfd-96d7-46154c6b2872 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:32:19.882Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=a30f74a3-eb5e-4cfd-96d7-46154c6b2872 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:32:20.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:32:20.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:32:21.927Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:32:10.630Z InvocationID=f4440787-6ba0-4b1a-9e12-001972981ba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:32:22.546Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ca45c71-43b2-4e16-acbf-6eb6919d8996 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:32:22.546Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ca45c71-43b2-4e16-acbf-6eb6919d8996 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@349d97e2 2022-04-26T06:32:22.546Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ca45c71-43b2-4e16-acbf-6eb6919d8996 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:32:22.546Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ca45c71-43b2-4e16-acbf-6eb6919d8996 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:32:23.561Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3ca45c71-43b2-4e16-acbf-6eb6919d8996 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:32:23.562Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3ca45c71-43b2-4e16-acbf-6eb6919d8996 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:32:24.335Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:32:10.630Z InvocationID=f4440787-6ba0-4b1a-9e12-001972981ba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:32:25.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f23c5a59-8d7d-43f1-9e20-20614bbfc787 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:32:25.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f23c5a59-8d7d-43f1-9e20-20614bbfc787 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@30b17ae 2022-04-26T06:32:25.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f23c5a59-8d7d-43f1-9e20-20614bbfc787 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:32:25.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f23c5a59-8d7d-43f1-9e20-20614bbfc787 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:32:26.477Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f23c5a59-8d7d-43f1-9e20-20614bbfc787 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:32:26.478Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f23c5a59-8d7d-43f1-9e20-20614bbfc787 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:32:26.740Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:32:10.630Z InvocationID=f4440787-6ba0-4b1a-9e12-001972981ba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:32:28.442Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d7f258d1-95e0-4774-ad4c-1bb0532117c0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:32:28.442Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d7f258d1-95e0-4774-ad4c-1bb0532117c0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7db98b1f 2022-04-26T06:32:28.443Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d7f258d1-95e0-4774-ad4c-1bb0532117c0 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:32:28.443Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d7f258d1-95e0-4774-ad4c-1bb0532117c0 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:32:29.481Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=d7f258d1-95e0-4774-ad4c-1bb0532117c0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:32:29.482Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=d7f258d1-95e0-4774-ad4c-1bb0532117c0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:32:29.544Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:32:10.630Z InvocationID=f4440787-6ba0-4b1a-9e12-001972981ba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:32:31.543Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f19646ad-f570-4459-8983-55583503b732 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:32:31.544Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f19646ad-f570-4459-8983-55583503b732 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1a2e9e5a 2022-04-26T06:32:31.544Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f19646ad-f570-4459-8983-55583503b732 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:32:31.544Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f19646ad-f570-4459-8983-55583503b732 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:32:32.350Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:32:10.630Z InvocationID=f4440787-6ba0-4b1a-9e12-001972981ba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:32:32.584Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f19646ad-f570-4459-8983-55583503b732 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:32:32.585Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f19646ad-f570-4459-8983-55583503b732 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:32:33.558Z [cluster2-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:32:33.559Z [cluster2-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 30000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:32:34.328Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed186b84-8b71-4c44-b43d-7d6921d0140b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:32:34.329Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed186b84-8b71-4c44-b43d-7d6921d0140b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@17d7a45b 2022-04-26T06:32:34.329Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed186b84-8b71-4c44-b43d-7d6921d0140b PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:32:34.329Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed186b84-8b71-4c44-b43d-7d6921d0140b PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:32:35.233Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:32:10.630Z InvocationID=f4440787-6ba0-4b1a-9e12-001972981ba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:32:35.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:32:35.370Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ed186b84-8b71-4c44-b43d-7d6921d0140b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:32:35.370Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ed186b84-8b71-4c44-b43d-7d6921d0140b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:32:35.379Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:32:37.433Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72509712-b6da-44bf-8af0-701afaaca771 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:32:37.433Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72509712-b6da-44bf-8af0-701afaaca771 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@22405c9f 2022-04-26T06:32:37.433Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72509712-b6da-44bf-8af0-701afaaca771 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:32:37.433Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72509712-b6da-44bf-8af0-701afaaca771 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:32:38.476Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=72509712-b6da-44bf-8af0-701afaaca771 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:32:38.477Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=72509712-b6da-44bf-8af0-701afaaca771 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:32:38.838Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:32:10.630Z InvocationID=f4440787-6ba0-4b1a-9e12-001972981ba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:32:39.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cea29bc3-c7ba-4b0e-b548-d03fb88cf8bb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:32:39.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cea29bc3-c7ba-4b0e-b548-d03fb88cf8bb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@17104ca4 2022-04-26T06:32:39.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cea29bc3-c7ba-4b0e-b548-d03fb88cf8bb PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:32:39.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cea29bc3-c7ba-4b0e-b548-d03fb88cf8bb PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:32:40.809Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=cea29bc3-c7ba-4b0e-b548-d03fb88cf8bb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:32:40.809Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=cea29bc3-c7ba-4b0e-b548-d03fb88cf8bb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:32:42.443Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:32:10.630Z InvocationID=f4440787-6ba0-4b1a-9e12-001972981ba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:32:42.786Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7f9067f-f3be-46d3-9fec-e2b292e65737 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:32:42.786Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7f9067f-f3be-46d3-9fec-e2b292e65737 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6d047237 2022-04-26T06:32:42.787Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7f9067f-f3be-46d3-9fec-e2b292e65737 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:32:42.787Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7f9067f-f3be-46d3-9fec-e2b292e65737 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:32:43.817Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e7f9067f-f3be-46d3-9fec-e2b292e65737 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:32:43.817Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e7f9067f-f3be-46d3-9fec-e2b292e65737 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:32:45.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=624a8cc4-ff12-481f-8909-9995223987dc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:32:45.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=624a8cc4-ff12-481f-8909-9995223987dc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3bf92f34 2022-04-26T06:32:45.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=624a8cc4-ff12-481f-8909-9995223987dc PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:32:45.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=624a8cc4-ff12-481f-8909-9995223987dc PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:32:46.048Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:32:10.630Z InvocationID=f4440787-6ba0-4b1a-9e12-001972981ba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:32:46.798Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=624a8cc4-ff12-481f-8909-9995223987dc ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:32:46.798Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=624a8cc4-ff12-481f-8909-9995223987dc ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:32:48.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72e93b45-ad03-4dca-8f41-8de73b66e1bb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:32:48.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72e93b45-ad03-4dca-8f41-8de73b66e1bb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3b31bf2c 2022-04-26T06:32:48.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72e93b45-ad03-4dca-8f41-8de73b66e1bb PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:32:48.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72e93b45-ad03-4dca-8f41-8de73b66e1bb PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:32:49.800Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=72e93b45-ad03-4dca-8f41-8de73b66e1bb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:32:49.801Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=72e93b45-ad03-4dca-8f41-8de73b66e1bb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:32:50.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:32:50.379Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:32:51.253Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:32:10.630Z InvocationID=f4440787-6ba0-4b1a-9e12-001972981ba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:32:52.537Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c4516ae4-9686-4e21-b8fc-fa4e8533067a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:32:52.537Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c4516ae4-9686-4e21-b8fc-fa4e8533067a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2e24679f 2022-04-26T06:32:52.538Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c4516ae4-9686-4e21-b8fc-fa4e8533067a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:32:52.538Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c4516ae4-9686-4e21-b8fc-fa4e8533067a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:32:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:32:52.686Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:32:52.686Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:32:53.578Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c4516ae4-9686-4e21-b8fc-fa4e8533067a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:32:53.579Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c4516ae4-9686-4e21-b8fc-fa4e8533067a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:32:55.440Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c14fab0e-b8ba-4486-9674-b5ba9b3efeb9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:32:55.440Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c14fab0e-b8ba-4486-9674-b5ba9b3efeb9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3ae163d9 2022-04-26T06:32:55.440Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c14fab0e-b8ba-4486-9674-b5ba9b3efeb9 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:32:55.440Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c14fab0e-b8ba-4486-9674-b5ba9b3efeb9 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:32:56.457Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c14fab0e-b8ba-4486-9674-b5ba9b3efeb9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:32:56.458Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c14fab0e-b8ba-4486-9674-b5ba9b3efeb9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:32:56.535Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:32:10.630Z InvocationID=f4440787-6ba0-4b1a-9e12-001972981ba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:32:58.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55b4ac02-4199-4fb2-b03f-ce0993a36c9d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:32:58.433Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55b4ac02-4199-4fb2-b03f-ce0993a36c9d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@58d0a190 2022-04-26T06:32:58.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55b4ac02-4199-4fb2-b03f-ce0993a36c9d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:32:58.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55b4ac02-4199-4fb2-b03f-ce0993a36c9d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:32:59.465Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=55b4ac02-4199-4fb2-b03f-ce0993a36c9d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:32:59.466Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=55b4ac02-4199-4fb2-b03f-ce0993a36c9d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:33:01.343Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=899710ce-bdc8-4a8b-a9fd-c8601574f856 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:33:01.344Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=899710ce-bdc8-4a8b-a9fd-c8601574f856 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1edbe40e 2022-04-26T06:33:01.344Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=899710ce-bdc8-4a8b-a9fd-c8601574f856 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:33:01.344Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=899710ce-bdc8-4a8b-a9fd-c8601574f856 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:33:02.377Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=899710ce-bdc8-4a8b-a9fd-c8601574f856 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:33:02.377Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=899710ce-bdc8-4a8b-a9fd-c8601574f856 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:33:04.332Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d70e4db2-c212-4581-af35-6ebb75bbfa34 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:33:04.332Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d70e4db2-c212-4581-af35-6ebb75bbfa34 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3f825ef0 2022-04-26T06:33:04.332Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d70e4db2-c212-4581-af35-6ebb75bbfa34 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:33:04.332Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d70e4db2-c212-4581-af35-6ebb75bbfa34 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:33:04.949Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:32:10.630Z InvocationID=f4440787-6ba0-4b1a-9e12-001972981ba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:33:05.358Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=d70e4db2-c212-4581-af35-6ebb75bbfa34 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:33:05.359Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=d70e4db2-c212-4581-af35-6ebb75bbfa34 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:33:05.368Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:33:05.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:33:07.328Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b548bef3-894d-487f-a044-a06433b40092 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:33:07.328Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b548bef3-894d-487f-a044-a06433b40092 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7c83ddf6 2022-04-26T06:33:07.328Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b548bef3-894d-487f-a044-a06433b40092 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:33:07.328Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b548bef3-894d-487f-a044-a06433b40092 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:33:07.661Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:33:08.365Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=b548bef3-894d-487f-a044-a06433b40092 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:33:08.366Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=b548bef3-894d-487f-a044-a06433b40092 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:33:09.797Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb843704-b46c-4f33-bd73-2ab7701cb93c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:33:09.797Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb843704-b46c-4f33-bd73-2ab7701cb93c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2e5f250e 2022-04-26T06:33:09.797Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb843704-b46c-4f33-bd73-2ab7701cb93c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:33:09.798Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb843704-b46c-4f33-bd73-2ab7701cb93c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:33:10.825Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=cb843704-b46c-4f33-bd73-2ab7701cb93c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:33:10.825Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=cb843704-b46c-4f33-bd73-2ab7701cb93c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:33:12.565Z [cluster2-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:33:12.566Z [cluster2-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 30000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:33:12.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c81c245c-fabb-42eb-bb2b-e930c737c292 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:33:12.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c81c245c-fabb-42eb-bb2b-e930c737c292 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3cb6c02f 2022-04-26T06:33:12.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c81c245c-fabb-42eb-bb2b-e930c737c292 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:33:12.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c81c245c-fabb-42eb-bb2b-e930c737c292 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:33:13.801Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c81c245c-fabb-42eb-bb2b-e930c737c292 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:33:13.801Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c81c245c-fabb-42eb-bb2b-e930c737c292 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:33:15.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ddf9c55f-f884-4fd8-b243-5fef1f56f30b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:33:15.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ddf9c55f-f884-4fd8-b243-5fef1f56f30b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@54dc9e43 2022-04-26T06:33:15.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ddf9c55f-f884-4fd8-b243-5fef1f56f30b PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:33:15.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ddf9c55f-f884-4fd8-b243-5fef1f56f30b PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:33:16.809Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ddf9c55f-f884-4fd8-b243-5fef1f56f30b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:33:16.809Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ddf9c55f-f884-4fd8-b243-5fef1f56f30b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:33:18.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a334af2-3ce8-41c1-abbb-488a7bd85c47 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:33:18.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a334af2-3ce8-41c1-abbb-488a7bd85c47 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@46123b7b 2022-04-26T06:33:18.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a334af2-3ce8-41c1-abbb-488a7bd85c47 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:33:18.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a334af2-3ce8-41c1-abbb-488a7bd85c47 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:33:19.754Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:32:10.630Z InvocationID=f4440787-6ba0-4b1a-9e12-001972981ba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:33:19.817Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=2a334af2-3ce8-41c1-abbb-488a7bd85c47 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:33:19.817Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=2a334af2-3ce8-41c1-abbb-488a7bd85c47 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:33:20.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:33:20.381Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:33:22.536Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=277e6949-46fa-4589-8a02-a7d11299a9bc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:33:22.536Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=277e6949-46fa-4589-8a02-a7d11299a9bc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@10751482 2022-04-26T06:33:22.537Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=277e6949-46fa-4589-8a02-a7d11299a9bc PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:33:22.537Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=277e6949-46fa-4589-8a02-a7d11299a9bc PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:33:23.561Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=277e6949-46fa-4589-8a02-a7d11299a9bc ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:33:23.562Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=277e6949-46fa-4589-8a02-a7d11299a9bc ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:33:25.338Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46b081db-f1ab-494a-b49a-9b64b0f60433 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:33:25.338Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46b081db-f1ab-494a-b49a-9b64b0f60433 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@721a8938 2022-04-26T06:33:25.338Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46b081db-f1ab-494a-b49a-9b64b0f60433 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:33:25.339Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46b081db-f1ab-494a-b49a-9b64b0f60433 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:33:26.377Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=46b081db-f1ab-494a-b49a-9b64b0f60433 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:33:26.377Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=46b081db-f1ab-494a-b49a-9b64b0f60433 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:33:28.341Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a7524e0b-33a3-4dc1-83f1-26fd51d81358 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:33:28.341Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a7524e0b-33a3-4dc1-83f1-26fd51d81358 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@37279236 2022-04-26T06:33:28.341Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a7524e0b-33a3-4dc1-83f1-26fd51d81358 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:33:28.341Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a7524e0b-33a3-4dc1-83f1-26fd51d81358 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:33:29.385Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=a7524e0b-33a3-4dc1-83f1-26fd51d81358 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:33:29.385Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=a7524e0b-33a3-4dc1-83f1-26fd51d81358 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:33:31.433Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14695dc7-ab04-487b-a4f8-530528a8cac1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:33:31.433Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14695dc7-ab04-487b-a4f8-530528a8cac1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@116e13e7 2022-04-26T06:33:31.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14695dc7-ab04-487b-a4f8-530528a8cac1 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:33:31.442Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14695dc7-ab04-487b-a4f8-530528a8cac1 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:33:32.474Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=14695dc7-ab04-487b-a4f8-530528a8cac1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:33:32.474Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=14695dc7-ab04-487b-a4f8-530528a8cac1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:33:34.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f627bb11-fffd-4358-88e5-118daeab2af7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:33:34.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f627bb11-fffd-4358-88e5-118daeab2af7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@67527a84 2022-04-26T06:33:34.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f627bb11-fffd-4358-88e5-118daeab2af7 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:33:34.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f627bb11-fffd-4358-88e5-118daeab2af7 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:33:34.560Z [qtp2085002312-17] ERROR o.j.g.database.StandardJanusGraph RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:32:10.630Z InvocationID=f4440787-6ba0-4b1a-9e12-001972981ba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Could not commit transaction [359] due to storage exception in commit org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT1M40S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 103 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 104 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:33:34.560Z [qtp2085002312-17] ERROR o.j.g.database.StandardJanusGraph RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:32:10.630Z InvocationID=f4440787-6ba0-4b1a-9e12-001972981ba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Could not commit transaction [359] due to exception org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT1M40S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 103 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 104 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:33:35.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:33:35.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:33:35.469Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f627bb11-fffd-4358-88e5-118daeab2af7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:33:35.469Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f627bb11-fffd-4358-88e5-118daeab2af7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:33:35.540Z [qtp2085002312-17] WARN o.o.sdc.be.servlets.ConsumerServlet RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:32:10.630Z InvocationID=f4440787-6ba0-4b1a-9e12-001972981ba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb #createConsumer('{"consumerName": "dcae", "consumerSalt": "9cd4c3ad2a6f6ce3f3414e68b5157e63", "consumerPassword": "35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495"}', 'Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/consumers)@6d0411d6', 'jh0003'): org.apache.cxf.jaxrs.impl.ResponseImpl@6c356dfa in 1.42min (too slow!) 2022-04-26T06:33:35.541Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=84911 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:32:10.630Z InvocationID=f4440787-6ba0-4b1a-9e12-001972981ba6 LogTimestamp=2022-04-26T06:33:35.541084Z ResponseDesc=Created Exiting. 2022-04-26T06:33:35.541Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:33:35.548Z [qtp2085002312-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:33:35.548522Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:33:35.548Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:33:35.548522Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:33:35.549Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermso HTTP/1.1 SC="500" 2022-04-26T06:33:35.555Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=7c33d85f-ca4e-41b6-88b3-091ef43a2959 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2022-04-26T06:33:35.555533Z InvocationID=fbee4464-ca0c-4507-8ebf-5f91e39d4b9f LogTimestamp=2022-04-26T06:33:35.555582Z Entering 2022-04-26T06:33:35.556Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=7c33d85f-ca4e-41b6-88b3-091ef43a2959 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2022-04-26T06:33:35.555533Z InvocationID=fbee4464-ca0c-4507-8ebf-5f91e39d4b9f LogTimestamp=2022-04-26T06:33:35.555582Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:33:35.558Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:33:35.556Z InvocationID=3a7f8c51-2a69-4b1b-af84-a5e173be1179 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id mso 2022-04-26T06:33:37.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a1ba8bf-be72-48c6-82c1-9eb72004049a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:33:37.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a1ba8bf-be72-48c6-82c1-9eb72004049a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@69cdfa7b 2022-04-26T06:33:37.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a1ba8bf-be72-48c6-82c1-9eb72004049a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:33:37.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a1ba8bf-be72-48c6-82c1-9eb72004049a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:33:38.235Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:33:35.556Z InvocationID=3a7f8c51-2a69-4b1b-af84-a5e173be1179 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:33:38.445Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=1a1ba8bf-be72-48c6-82c1-9eb72004049a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:33:38.445Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=1a1ba8bf-be72-48c6-82c1-9eb72004049a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:33:39.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ceff0940-a8b4-4f24-9d2d-b11cb7bf6b62 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:33:39.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ceff0940-a8b4-4f24-9d2d-b11cb7bf6b62 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@edbd8cf 2022-04-26T06:33:39.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ceff0940-a8b4-4f24-9d2d-b11cb7bf6b62 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:33:39.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ceff0940-a8b4-4f24-9d2d-b11cb7bf6b62 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:33:40.291Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:33:35.556Z InvocationID=3a7f8c51-2a69-4b1b-af84-a5e173be1179 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:33:40.809Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ceff0940-a8b4-4f24-9d2d-b11cb7bf6b62 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:33:40.810Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ceff0940-a8b4-4f24-9d2d-b11cb7bf6b62 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:33:42.347Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:33:35.556Z InvocationID=3a7f8c51-2a69-4b1b-af84-a5e173be1179 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:33:42.790Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ef7d2d1-1334-4ea4-97f2-fa5e3f19aa5c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:33:42.791Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ef7d2d1-1334-4ea4-97f2-fa5e3f19aa5c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6972695d 2022-04-26T06:33:42.791Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ef7d2d1-1334-4ea4-97f2-fa5e3f19aa5c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:33:42.791Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ef7d2d1-1334-4ea4-97f2-fa5e3f19aa5c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:33:43.821Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=0ef7d2d1-1334-4ea4-97f2-fa5e3f19aa5c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:33:43.821Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=0ef7d2d1-1334-4ea4-97f2-fa5e3f19aa5c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:33:44.452Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:33:35.556Z InvocationID=3a7f8c51-2a69-4b1b-af84-a5e173be1179 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:33:45.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96e5069f-59ab-4b35-a536-88301c9c69a9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:33:45.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96e5069f-59ab-4b35-a536-88301c9c69a9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@bfe0a05 2022-04-26T06:33:45.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96e5069f-59ab-4b35-a536-88301c9c69a9 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:33:45.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96e5069f-59ab-4b35-a536-88301c9c69a9 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:33:46.659Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:33:35.556Z InvocationID=3a7f8c51-2a69-4b1b-af84-a5e173be1179 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:33:46.794Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=96e5069f-59ab-4b35-a536-88301c9c69a9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:33:46.795Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=96e5069f-59ab-4b35-a536-88301c9c69a9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:33:48.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf02b4ba-52b3-4344-9ea5-fee1201678d0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:33:48.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf02b4ba-52b3-4344-9ea5-fee1201678d0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4cac0912 2022-04-26T06:33:48.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf02b4ba-52b3-4344-9ea5-fee1201678d0 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:33:48.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf02b4ba-52b3-4344-9ea5-fee1201678d0 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:33:48.864Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:33:35.556Z InvocationID=3a7f8c51-2a69-4b1b-af84-a5e173be1179 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:33:49.561Z [cluster2-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:33:49.562Z [cluster2-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 30000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:33:49.801Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=bf02b4ba-52b3-4344-9ea5-fee1201678d0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:33:49.802Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=bf02b4ba-52b3-4344-9ea5-fee1201678d0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:33:50.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:33:50.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:33:51.130Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:33:35.556Z InvocationID=3a7f8c51-2a69-4b1b-af84-a5e173be1179 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:33:52.531Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e473a66-d58d-4a47-bfd1-1a04c7368a22 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:33:52.531Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e473a66-d58d-4a47-bfd1-1a04c7368a22 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@71ec12b7 2022-04-26T06:33:52.532Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e473a66-d58d-4a47-bfd1-1a04c7368a22 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:33:52.532Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e473a66-d58d-4a47-bfd1-1a04c7368a22 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:33:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:33:52.686Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:33:52.686Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:33:53.535Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:33:35.556Z InvocationID=3a7f8c51-2a69-4b1b-af84-a5e173be1179 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:33:53.545Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=1e473a66-d58d-4a47-bfd1-1a04c7368a22 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:33:53.545Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=1e473a66-d58d-4a47-bfd1-1a04c7368a22 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:33:55.431Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d2fd6f9b-2886-4064-9072-fe9b1054c173 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:33:55.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d2fd6f9b-2886-4064-9072-fe9b1054c173 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1eb99772 2022-04-26T06:33:55.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d2fd6f9b-2886-4064-9072-fe9b1054c173 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:33:55.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d2fd6f9b-2886-4064-9072-fe9b1054c173 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:33:56.340Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:33:35.556Z InvocationID=3a7f8c51-2a69-4b1b-af84-a5e173be1179 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:33:56.457Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=d2fd6f9b-2886-4064-9072-fe9b1054c173 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:33:56.457Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=d2fd6f9b-2886-4064-9072-fe9b1054c173 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:33:58.341Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=abc9fefb-2950-4048-9ffa-1c37bef33a30 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:33:58.342Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=abc9fefb-2950-4048-9ffa-1c37bef33a30 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@49503c4f 2022-04-26T06:33:58.342Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=abc9fefb-2950-4048-9ffa-1c37bef33a30 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:33:58.342Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=abc9fefb-2950-4048-9ffa-1c37bef33a30 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:33:59.145Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:33:35.556Z InvocationID=3a7f8c51-2a69-4b1b-af84-a5e173be1179 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:33:59.369Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=abc9fefb-2950-4048-9ffa-1c37bef33a30 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:33:59.369Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=abc9fefb-2950-4048-9ffa-1c37bef33a30 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:34:01.433Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=519f41bc-9634-4e12-a581-27a91484bbbf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:34:01.433Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=519f41bc-9634-4e12-a581-27a91484bbbf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@26296eec 2022-04-26T06:34:01.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=519f41bc-9634-4e12-a581-27a91484bbbf PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:34:01.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=519f41bc-9634-4e12-a581-27a91484bbbf PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:34:02.031Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:33:35.556Z InvocationID=3a7f8c51-2a69-4b1b-af84-a5e173be1179 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:34:02.473Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=519f41bc-9634-4e12-a581-27a91484bbbf ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:34:02.474Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=519f41bc-9634-4e12-a581-27a91484bbbf ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:34:04.532Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6364dd58-d542-402b-8748-bf8a47d6f5d0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:34:04.532Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6364dd58-d542-402b-8748-bf8a47d6f5d0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@71565d9e 2022-04-26T06:34:04.532Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6364dd58-d542-402b-8748-bf8a47d6f5d0 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:34:04.533Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6364dd58-d542-402b-8748-bf8a47d6f5d0 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:34:05.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:34:05.377Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:34:05.577Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=6364dd58-d542-402b-8748-bf8a47d6f5d0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:34:05.577Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=6364dd58-d542-402b-8748-bf8a47d6f5d0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:34:05.636Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:33:35.556Z InvocationID=3a7f8c51-2a69-4b1b-af84-a5e173be1179 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:34:07.344Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5dccd510-838c-4214-95f7-e817d4db335b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:34:07.345Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5dccd510-838c-4214-95f7-e817d4db335b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@230b1e52 2022-04-26T06:34:07.345Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5dccd510-838c-4214-95f7-e817d4db335b PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:34:07.345Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5dccd510-838c-4214-95f7-e817d4db335b PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:34:07.654Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:34:08.362Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=5dccd510-838c-4214-95f7-e817d4db335b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:34:08.362Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=5dccd510-838c-4214-95f7-e817d4db335b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:34:09.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52784a84-fdc9-49fa-830e-f7b045ae4952 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:34:09.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52784a84-fdc9-49fa-830e-f7b045ae4952 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3c67e76f 2022-04-26T06:34:09.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52784a84-fdc9-49fa-830e-f7b045ae4952 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:34:09.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52784a84-fdc9-49fa-830e-f7b045ae4952 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:34:10.793Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=52784a84-fdc9-49fa-830e-f7b045ae4952 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:34:10.793Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=52784a84-fdc9-49fa-830e-f7b045ae4952 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:34:10.843Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:33:35.556Z InvocationID=3a7f8c51-2a69-4b1b-af84-a5e173be1179 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:34:12.833Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9cb83fa4-9b6f-4db3-8ae5-261a927b6f4f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:34:12.833Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9cb83fa4-9b6f-4db3-8ae5-261a927b6f4f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4b6a7d2a 2022-04-26T06:34:12.833Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9cb83fa4-9b6f-4db3-8ae5-261a927b6f4f PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:34:12.833Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9cb83fa4-9b6f-4db3-8ae5-261a927b6f4f PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:34:13.865Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=9cb83fa4-9b6f-4db3-8ae5-261a927b6f4f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:34:13.866Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=9cb83fa4-9b6f-4db3-8ae5-261a927b6f4f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:34:15.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bbcbbad-0293-4a92-b601-0df541b11383 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:34:15.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bbcbbad-0293-4a92-b601-0df541b11383 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@40de4b6f 2022-04-26T06:34:15.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bbcbbad-0293-4a92-b601-0df541b11383 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:34:15.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bbcbbad-0293-4a92-b601-0df541b11383 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:34:16.049Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:33:35.556Z InvocationID=3a7f8c51-2a69-4b1b-af84-a5e173be1179 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:34:16.809Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3bbcbbad-0293-4a92-b601-0df541b11383 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:34:16.810Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3bbcbbad-0293-4a92-b601-0df541b11383 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:34:18.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c4e39cf2-ca47-49c9-a1c7-bb7cb000a275 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:34:18.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c4e39cf2-ca47-49c9-a1c7-bb7cb000a275 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@28ba3048 2022-04-26T06:34:18.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c4e39cf2-ca47-49c9-a1c7-bb7cb000a275 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:34:18.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c4e39cf2-ca47-49c9-a1c7-bb7cb000a275 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:34:19.817Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c4e39cf2-ca47-49c9-a1c7-bb7cb000a275 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:34:19.818Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c4e39cf2-ca47-49c9-a1c7-bb7cb000a275 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:34:20.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:34:20.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:34:22.531Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:33:35.556Z InvocationID=3a7f8c51-2a69-4b1b-af84-a5e173be1179 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:34:22.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7c77931-b2d3-476e-9049-007d0e247d44 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:34:22.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7c77931-b2d3-476e-9049-007d0e247d44 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5544f17a 2022-04-26T06:34:22.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7c77931-b2d3-476e-9049-007d0e247d44 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:34:22.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7c77931-b2d3-476e-9049-007d0e247d44 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:34:23.657Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c7c77931-b2d3-476e-9049-007d0e247d44 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:34:23.658Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c7c77931-b2d3-476e-9049-007d0e247d44 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:34:25.528Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc1a19e6-2ef3-46b8-8990-4516fd363c56 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:34:25.528Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc1a19e6-2ef3-46b8-8990-4516fd363c56 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@10da1dc2 2022-04-26T06:34:25.528Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc1a19e6-2ef3-46b8-8990-4516fd363c56 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:34:25.528Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc1a19e6-2ef3-46b8-8990-4516fd363c56 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:34:26.569Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=cc1a19e6-2ef3-46b8-8990-4516fd363c56 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:34:26.569Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=cc1a19e6-2ef3-46b8-8990-4516fd363c56 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:34:28.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f98e6f43-a670-48b6-b00d-9ca58645732b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:34:28.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f98e6f43-a670-48b6-b00d-9ca58645732b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f37ca60 2022-04-26T06:34:28.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f98e6f43-a670-48b6-b00d-9ca58645732b PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:34:28.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f98e6f43-a670-48b6-b00d-9ca58645732b PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:34:29.455Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f98e6f43-a670-48b6-b00d-9ca58645732b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:34:29.456Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f98e6f43-a670-48b6-b00d-9ca58645732b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:34:31.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70483ec8-9bfc-47f9-ace0-f03fff2da670 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:34:31.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70483ec8-9bfc-47f9-ace0-f03fff2da670 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6cad289b 2022-04-26T06:34:31.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70483ec8-9bfc-47f9-ace0-f03fff2da670 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:34:31.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70483ec8-9bfc-47f9-ace0-f03fff2da670 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:34:32.650Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=70483ec8-9bfc-47f9-ace0-f03fff2da670 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:34:32.650Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=70483ec8-9bfc-47f9-ace0-f03fff2da670 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:34:34.738Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=565bcf97-fff2-4a2b-91bb-3dc17aeeaefd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:34:34.739Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=565bcf97-fff2-4a2b-91bb-3dc17aeeaefd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7112be4d 2022-04-26T06:34:34.739Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=565bcf97-fff2-4a2b-91bb-3dc17aeeaefd PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:34:34.739Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=565bcf97-fff2-4a2b-91bb-3dc17aeeaefd PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:34:35.340Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:33:35.556Z InvocationID=3a7f8c51-2a69-4b1b-af84-a5e173be1179 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:34:35.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:34:35.393Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:34:35.753Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=565bcf97-fff2-4a2b-91bb-3dc17aeeaefd ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:34:35.753Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=565bcf97-fff2-4a2b-91bb-3dc17aeeaefd ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:34:37.428Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee68f4c9-7ca4-47f7-87f2-7272bfb11cb0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:34:37.428Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee68f4c9-7ca4-47f7-87f2-7272bfb11cb0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@10e52886 2022-04-26T06:34:37.428Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee68f4c9-7ca4-47f7-87f2-7272bfb11cb0 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:34:37.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee68f4c9-7ca4-47f7-87f2-7272bfb11cb0 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:34:38.447Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ee68f4c9-7ca4-47f7-87f2-7272bfb11cb0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:34:38.447Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ee68f4c9-7ca4-47f7-87f2-7272bfb11cb0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:34:39.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73855a9e-e357-4496-8535-c6374bb94525 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:34:39.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73855a9e-e357-4496-8535-c6374bb94525 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1c09719f 2022-04-26T06:34:39.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73855a9e-e357-4496-8535-c6374bb94525 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:34:39.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73855a9e-e357-4496-8535-c6374bb94525 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:34:40.809Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=73855a9e-e357-4496-8535-c6374bb94525 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:34:40.809Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=73855a9e-e357-4496-8535-c6374bb94525 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:34:42.788Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fcb4daf3-c32b-4d7c-baef-cac9fb436010 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:34:42.788Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fcb4daf3-c32b-4d7c-baef-cac9fb436010 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@79de8397 2022-04-26T06:34:42.788Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fcb4daf3-c32b-4d7c-baef-cac9fb436010 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:34:42.788Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fcb4daf3-c32b-4d7c-baef-cac9fb436010 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:34:43.817Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=fcb4daf3-c32b-4d7c-baef-cac9fb436010 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:34:43.817Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=fcb4daf3-c32b-4d7c-baef-cac9fb436010 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:34:45.829Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0949582-0ee5-41ab-88e9-06deb6b9d23f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:34:45.829Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0949582-0ee5-41ab-88e9-06deb6b9d23f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3c53203f 2022-04-26T06:34:45.829Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0949582-0ee5-41ab-88e9-06deb6b9d23f PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:34:45.829Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0949582-0ee5-41ab-88e9-06deb6b9d23f PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:34:46.857Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c0949582-0ee5-41ab-88e9-06deb6b9d23f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:34:46.858Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c0949582-0ee5-41ab-88e9-06deb6b9d23f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:34:48.149Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:33:35.556Z InvocationID=3a7f8c51-2a69-4b1b-af84-a5e173be1179 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:34:48.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25f4ad4d-6e9b-413c-a314-a5c13b1948d8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:34:48.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25f4ad4d-6e9b-413c-a314-a5c13b1948d8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@40bb3b1c 2022-04-26T06:34:48.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25f4ad4d-6e9b-413c-a314-a5c13b1948d8 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:34:48.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25f4ad4d-6e9b-413c-a314-a5c13b1948d8 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:34:49.801Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=25f4ad4d-6e9b-413c-a314-a5c13b1948d8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:34:49.801Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=25f4ad4d-6e9b-413c-a314-a5c13b1948d8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:34:50.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:34:50.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:34:52.531Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c7c0a98-e195-4837-a60c-f8853a8c2c42 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:34:52.531Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c7c0a98-e195-4837-a60c-f8853a8c2c42 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7a5df96 2022-04-26T06:34:52.531Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c7c0a98-e195-4837-a60c-f8853a8c2c42 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:34:52.532Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c7c0a98-e195-4837-a60c-f8853a8c2c42 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:34:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:34:52.687Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:34:52.687Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:34:53.546Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=6c7c0a98-e195-4837-a60c-f8853a8c2c42 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:34:53.547Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=6c7c0a98-e195-4837-a60c-f8853a8c2c42 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:34:55.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe046f74-4747-4bdd-ba7b-88d3428a04f7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:34:55.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe046f74-4747-4bdd-ba7b-88d3428a04f7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7518ed1c 2022-04-26T06:34:55.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe046f74-4747-4bdd-ba7b-88d3428a04f7 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:34:55.430Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe046f74-4747-4bdd-ba7b-88d3428a04f7 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:34:56.457Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=fe046f74-4747-4bdd-ba7b-88d3428a04f7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:34:56.459Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=fe046f74-4747-4bdd-ba7b-88d3428a04f7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:34:58.436Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25161046-2d59-4098-8809-219a02d87ca9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:34:58.436Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25161046-2d59-4098-8809-219a02d87ca9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6c031fca 2022-04-26T06:34:58.436Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25161046-2d59-4098-8809-219a02d87ca9 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:34:58.436Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25161046-2d59-4098-8809-219a02d87ca9 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:34:59.470Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=25161046-2d59-4098-8809-219a02d87ca9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:34:59.471Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=25161046-2d59-4098-8809-219a02d87ca9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:35:01.331Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ecc8bc8f-f945-4d1e-8460-a0e49d4db2b0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:35:01.332Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ecc8bc8f-f945-4d1e-8460-a0e49d4db2b0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@209fada7 2022-04-26T06:35:01.332Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ecc8bc8f-f945-4d1e-8460-a0e49d4db2b0 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:35:01.332Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ecc8bc8f-f945-4d1e-8460-a0e49d4db2b0 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:35:02.350Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ecc8bc8f-f945-4d1e-8460-a0e49d4db2b0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:35:02.351Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ecc8bc8f-f945-4d1e-8460-a0e49d4db2b0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:35:04.437Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55e174bd-3d3e-44f8-b2ae-96e80c034fa2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:35:04.437Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55e174bd-3d3e-44f8-b2ae-96e80c034fa2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8353272 2022-04-26T06:35:04.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55e174bd-3d3e-44f8-b2ae-96e80c034fa2 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:35:04.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55e174bd-3d3e-44f8-b2ae-96e80c034fa2 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:35:05.369Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:35:05.377Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:35:05.481Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=55e174bd-3d3e-44f8-b2ae-96e80c034fa2 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:35:05.482Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=55e174bd-3d3e-44f8-b2ae-96e80c034fa2 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:35:07.534Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d99c7db3-0f51-4133-b6f4-1398d6eff7ab PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:35:07.534Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d99c7db3-0f51-4133-b6f4-1398d6eff7ab PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@292c3a91 2022-04-26T06:35:07.534Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d99c7db3-0f51-4133-b6f4-1398d6eff7ab PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:35:07.534Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d99c7db3-0f51-4133-b6f4-1398d6eff7ab PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:35:07.690Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:35:08.553Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=d99c7db3-0f51-4133-b6f4-1398d6eff7ab ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:35:08.553Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=d99c7db3-0f51-4133-b6f4-1398d6eff7ab ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:35:09.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=731fcca0-bc22-4538-9a32-7f7d1313d984 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:35:09.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=731fcca0-bc22-4538-9a32-7f7d1313d984 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@10f4e5b5 2022-04-26T06:35:09.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=731fcca0-bc22-4538-9a32-7f7d1313d984 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:35:09.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=731fcca0-bc22-4538-9a32-7f7d1313d984 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:35:10.793Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=731fcca0-bc22-4538-9a32-7f7d1313d984 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:35:10.794Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=731fcca0-bc22-4538-9a32-7f7d1313d984 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:35:12.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5f01342-6860-4123-80a3-5c325933a476 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:35:12.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5f01342-6860-4123-80a3-5c325933a476 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@36b0f3b0 2022-04-26T06:35:12.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5f01342-6860-4123-80a3-5c325933a476 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:35:12.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5f01342-6860-4123-80a3-5c325933a476 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:35:13.759Z [qtp2085002312-20] ERROR o.j.g.database.StandardJanusGraph RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:33:35.556Z InvocationID=3a7f8c51-2a69-4b1b-af84-a5e173be1179 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Could not commit transaction [362] due to storage exception in commit org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT1M40S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 103 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 104 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:35:13.760Z [qtp2085002312-20] ERROR o.j.g.database.StandardJanusGraph RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:33:35.556Z InvocationID=3a7f8c51-2a69-4b1b-af84-a5e173be1179 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Could not commit transaction [362] due to exception org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT1M40S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 103 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 104 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:35:13.805Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e5f01342-6860-4123-80a3-5c325933a476 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:35:13.805Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e5f01342-6860-4123-80a3-5c325933a476 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:35:14.349Z [qtp2085002312-20] WARN o.o.sdc.be.servlets.ConsumerServlet RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:33:35.556Z InvocationID=3a7f8c51-2a69-4b1b-af84-a5e173be1179 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb #createConsumer('{"consumerName": "mso", "consumerSalt": "9cd4c3ad2a6f6ce3f3414e68b5157e63", "consumerPassword": "35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495"}', 'Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/consumers)@6d0411d6', 'jh0003'): org.apache.cxf.jaxrs.impl.ResponseImpl@68437802 in 1.65min (too slow!) 2022-04-26T06:35:14.352Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=98795 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:33:35.556Z InvocationID=3a7f8c51-2a69-4b1b-af84-a5e173be1179 LogTimestamp=2022-04-26T06:35:14.351169Z ResponseDesc=Created Exiting. 2022-04-26T06:35:14.352Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:35:14.368Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=116496 PartnerName=kube-probe/1.22 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=ERROR ServerIPAddress=10.233.65.41 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2022-04-26T06:33:17.872Z InvocationID=811c4276-bca0-4a77-943c-eba05ca947e0 ErrorCode=500 LogTimestamp=2022-04-26T06:35:14.368379Z ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:35:14.369Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usersdnc HTTP/1.1 SC="500" 2022-04-26T06:35:14.375Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=e79bed78-f967-47b5-a137-90f32d21af32 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2022-04-26T06:35:14.375658Z InvocationID=93c6089e-5e8d-4a28-9dda-9b83c9f7955b LogTimestamp=2022-04-26T06:35:14.375716Z Entering 2022-04-26T06:35:14.376Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=e79bed78-f967-47b5-a137-90f32d21af32 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2022-04-26T06:35:14.375658Z InvocationID=93c6089e-5e8d-4a28-9dda-9b83c9f7955b LogTimestamp=2022-04-26T06:35:14.375716Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:35:14.380Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:35:14.376Z InvocationID=5a7d4e26-0c41-446f-ae06-443617ed99d9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id sdnc 2022-04-26T06:35:14.916Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:35:14.376Z InvocationID=5a7d4e26-0c41-446f-ae06-443617ed99d9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:35:14.973Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:35:14.376Z InvocationID=5a7d4e26-0c41-446f-ae06-443617ed99d9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:35:15.081Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:35:14.376Z InvocationID=5a7d4e26-0c41-446f-ae06-443617ed99d9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:35:15.291Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:35:14.376Z InvocationID=5a7d4e26-0c41-446f-ae06-443617ed99d9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:35:15.497Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:35:14.376Z InvocationID=5a7d4e26-0c41-446f-ae06-443617ed99d9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:35:15.708Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:35:14.376Z InvocationID=5a7d4e26-0c41-446f-ae06-443617ed99d9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:35:15.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0475d8f3-a429-4285-bf2f-d691d88bec94 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:35:15.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0475d8f3-a429-4285-bf2f-d691d88bec94 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2fb90470 2022-04-26T06:35:15.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0475d8f3-a429-4285-bf2f-d691d88bec94 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:35:15.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0475d8f3-a429-4285-bf2f-d691d88bec94 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:35:16.115Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:35:14.376Z InvocationID=5a7d4e26-0c41-446f-ae06-443617ed99d9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:35:16.809Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=0475d8f3-a429-4285-bf2f-d691d88bec94 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:35:16.810Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=0475d8f3-a429-4285-bf2f-d691d88bec94 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:35:16.922Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:35:14.376Z InvocationID=5a7d4e26-0c41-446f-ae06-443617ed99d9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:35:18.530Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:35:14.376Z InvocationID=5a7d4e26-0c41-446f-ae06-443617ed99d9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:35:18.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3028566a-dca3-4d80-aa3f-4bfd4fc9234a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:35:18.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3028566a-dca3-4d80-aa3f-4bfd4fc9234a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@65ecc937 2022-04-26T06:35:18.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3028566a-dca3-4d80-aa3f-4bfd4fc9234a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:35:18.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3028566a-dca3-4d80-aa3f-4bfd4fc9234a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:35:19.817Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3028566a-dca3-4d80-aa3f-4bfd4fc9234a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:35:19.817Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3028566a-dca3-4d80-aa3f-4bfd4fc9234a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:35:20.142Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:35:14.376Z InvocationID=5a7d4e26-0c41-446f-ae06-443617ed99d9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:35:20.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:35:20.388Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:35:22.528Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a80d5745-ce40-4140-80e7-5399eac24c98 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:35:22.529Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a80d5745-ce40-4140-80e7-5399eac24c98 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@267e0b5f 2022-04-26T06:35:22.529Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a80d5745-ce40-4140-80e7-5399eac24c98 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:35:22.529Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a80d5745-ce40-4140-80e7-5399eac24c98 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:35:23.349Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:35:14.376Z InvocationID=5a7d4e26-0c41-446f-ae06-443617ed99d9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:35:23.565Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=a80d5745-ce40-4140-80e7-5399eac24c98 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:35:23.565Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=a80d5745-ce40-4140-80e7-5399eac24c98 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:35:25.427Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=172e24a3-83ce-47a1-9f8d-be36a3b3d360 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:35:25.427Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=172e24a3-83ce-47a1-9f8d-be36a3b3d360 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@759393b7 2022-04-26T06:35:25.427Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=172e24a3-83ce-47a1-9f8d-be36a3b3d360 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:35:25.428Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=172e24a3-83ce-47a1-9f8d-be36a3b3d360 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:35:26.441Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=172e24a3-83ce-47a1-9f8d-be36a3b3d360 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:35:26.441Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=172e24a3-83ce-47a1-9f8d-be36a3b3d360 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:35:26.631Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:35:14.376Z InvocationID=5a7d4e26-0c41-446f-ae06-443617ed99d9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:35:28.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0c7bf6b-dc43-40b7-942e-d4b0154f5db8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:35:28.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0c7bf6b-dc43-40b7-942e-d4b0154f5db8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7ad76573 2022-04-26T06:35:28.430Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0c7bf6b-dc43-40b7-942e-d4b0154f5db8 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:35:28.430Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0c7bf6b-dc43-40b7-942e-d4b0154f5db8 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:35:29.454Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f0c7bf6b-dc43-40b7-942e-d4b0154f5db8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:35:29.454Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f0c7bf6b-dc43-40b7-942e-d4b0154f5db8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:35:29.844Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:35:14.376Z InvocationID=5a7d4e26-0c41-446f-ae06-443617ed99d9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:35:31.533Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=35126dd4-dac9-4b4c-82d7-341cf139b33d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:35:31.534Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=35126dd4-dac9-4b4c-82d7-341cf139b33d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4f10562d 2022-04-26T06:35:31.534Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=35126dd4-dac9-4b4c-82d7-341cf139b33d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:35:31.534Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=35126dd4-dac9-4b4c-82d7-341cf139b33d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:35:32.553Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=35126dd4-dac9-4b4c-82d7-341cf139b33d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:35:32.554Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=35126dd4-dac9-4b4c-82d7-341cf139b33d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:35:34.442Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=270f7dee-87e6-478b-baf3-20dc99c65d46 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:35:34.445Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=270f7dee-87e6-478b-baf3-20dc99c65d46 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@fb32734 2022-04-26T06:35:34.445Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=270f7dee-87e6-478b-baf3-20dc99c65d46 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:35:34.446Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=270f7dee-87e6-478b-baf3-20dc99c65d46 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:35:35.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:35:35.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:35:35.469Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=270f7dee-87e6-478b-baf3-20dc99c65d46 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:35:35.469Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=270f7dee-87e6-478b-baf3-20dc99c65d46 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:35:36.341Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:35:14.376Z InvocationID=5a7d4e26-0c41-446f-ae06-443617ed99d9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:35:37.339Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=854c96b0-a62f-43f2-9574-73c7b3030b0f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:35:37.340Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=854c96b0-a62f-43f2-9574-73c7b3030b0f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5f0e68ed 2022-04-26T06:35:37.340Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=854c96b0-a62f-43f2-9574-73c7b3030b0f PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:35:37.340Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=854c96b0-a62f-43f2-9574-73c7b3030b0f PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:35:38.383Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=854c96b0-a62f-43f2-9574-73c7b3030b0f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:35:38.384Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=854c96b0-a62f-43f2-9574-73c7b3030b0f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:35:39.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7ea6cf8-1225-4d66-b2c6-e4922402e5ff PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:35:39.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7ea6cf8-1225-4d66-b2c6-e4922402e5ff PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@76d8602b 2022-04-26T06:35:39.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7ea6cf8-1225-4d66-b2c6-e4922402e5ff PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:35:39.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7ea6cf8-1225-4d66-b2c6-e4922402e5ff PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:35:40.809Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=b7ea6cf8-1225-4d66-b2c6-e4922402e5ff ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:35:40.809Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=b7ea6cf8-1225-4d66-b2c6-e4922402e5ff ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:35:42.751Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:35:14.376Z InvocationID=5a7d4e26-0c41-446f-ae06-443617ed99d9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:35:42.788Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ffc8620-6715-44c0-92ea-3c181ec10777 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:35:42.788Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ffc8620-6715-44c0-92ea-3c181ec10777 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5f7703b8 2022-04-26T06:35:42.788Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ffc8620-6715-44c0-92ea-3c181ec10777 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:35:42.788Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ffc8620-6715-44c0-92ea-3c181ec10777 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:35:43.817Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=1ffc8620-6715-44c0-92ea-3c181ec10777 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:35:43.817Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=1ffc8620-6715-44c0-92ea-3c181ec10777 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:35:45.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37acbda7-64f5-4584-8fdd-f872a0ce5001 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:35:45.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37acbda7-64f5-4584-8fdd-f872a0ce5001 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2990c3ab 2022-04-26T06:35:45.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37acbda7-64f5-4584-8fdd-f872a0ce5001 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:35:45.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37acbda7-64f5-4584-8fdd-f872a0ce5001 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:35:46.796Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=37acbda7-64f5-4584-8fdd-f872a0ce5001 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:35:46.797Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=37acbda7-64f5-4584-8fdd-f872a0ce5001 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:35:48.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11a2b291-c1ea-49a0-8693-e7f2e697371f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:35:48.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11a2b291-c1ea-49a0-8693-e7f2e697371f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2fcfb921 2022-04-26T06:35:48.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11a2b291-c1ea-49a0-8693-e7f2e697371f PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:35:48.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11a2b291-c1ea-49a0-8693-e7f2e697371f PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:35:49.159Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:35:14.376Z InvocationID=5a7d4e26-0c41-446f-ae06-443617ed99d9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:35:49.801Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=11a2b291-c1ea-49a0-8693-e7f2e697371f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:35:49.801Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=11a2b291-c1ea-49a0-8693-e7f2e697371f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:35:50.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:35:50.377Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:35:52.539Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7da5b746-c0b2-4e22-b916-d2808d90fcff PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:35:52.539Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7da5b746-c0b2-4e22-b916-d2808d90fcff PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7de5dc1e 2022-04-26T06:35:52.539Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7da5b746-c0b2-4e22-b916-d2808d90fcff PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:35:52.539Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7da5b746-c0b2-4e22-b916-d2808d90fcff PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:35:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:35:52.686Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:35:52.686Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:35:53.577Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=7da5b746-c0b2-4e22-b916-d2808d90fcff ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:35:53.577Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=7da5b746-c0b2-4e22-b916-d2808d90fcff ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:35:55.433Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=678bba44-300b-4b20-959d-ca02b820f9a6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:35:55.433Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=678bba44-300b-4b20-959d-ca02b820f9a6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@77e1614f 2022-04-26T06:35:55.433Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=678bba44-300b-4b20-959d-ca02b820f9a6 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:35:55.433Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=678bba44-300b-4b20-959d-ca02b820f9a6 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:35:55.631Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:35:14.376Z InvocationID=5a7d4e26-0c41-446f-ae06-443617ed99d9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:35:56.457Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=678bba44-300b-4b20-959d-ca02b820f9a6 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:35:56.458Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=678bba44-300b-4b20-959d-ca02b820f9a6 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:35:58.528Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=13c0fd43-cf5a-4d95-b394-2e158be6fb13 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:35:58.528Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=13c0fd43-cf5a-4d95-b394-2e158be6fb13 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@481f717e 2022-04-26T06:35:58.528Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=13c0fd43-cf5a-4d95-b394-2e158be6fb13 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:35:58.528Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=13c0fd43-cf5a-4d95-b394-2e158be6fb13 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:35:59.561Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=13c0fd43-cf5a-4d95-b394-2e158be6fb13 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:35:59.561Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=13c0fd43-cf5a-4d95-b394-2e158be6fb13 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:36:01.528Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d79364a0-1f6f-49fd-a5fe-4090b8af899a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:36:01.529Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d79364a0-1f6f-49fd-a5fe-4090b8af899a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3fa5ae3e 2022-04-26T06:36:01.529Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d79364a0-1f6f-49fd-a5fe-4090b8af899a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:36:01.529Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d79364a0-1f6f-49fd-a5fe-4090b8af899a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:36:02.569Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=d79364a0-1f6f-49fd-a5fe-4090b8af899a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:36:02.570Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=d79364a0-1f6f-49fd-a5fe-4090b8af899a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:36:04.331Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4b66374-1d69-4e59-b961-39f5caedbb28 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:36:04.331Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4b66374-1d69-4e59-b961-39f5caedbb28 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5dc3d23f 2022-04-26T06:36:04.331Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4b66374-1d69-4e59-b961-39f5caedbb28 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:36:04.332Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4b66374-1d69-4e59-b961-39f5caedbb28 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:36:05.353Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e4b66374-1d69-4e59-b961-39f5caedbb28 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:36:05.353Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e4b66374-1d69-4e59-b961-39f5caedbb28 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:36:05.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:36:05.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:36:07.341Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aac49c4c-c059-49a6-ae61-708f765dbacc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:36:07.342Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aac49c4c-c059-49a6-ae61-708f765dbacc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1e46f45d 2022-04-26T06:36:07.342Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aac49c4c-c059-49a6-ae61-708f765dbacc PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:36:07.342Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aac49c4c-c059-49a6-ae61-708f765dbacc PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:36:07.659Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:36:08.361Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=aac49c4c-c059-49a6-ae61-708f765dbacc ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:36:08.361Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=aac49c4c-c059-49a6-ae61-708f765dbacc ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:36:08.437Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:35:14.376Z InvocationID=5a7d4e26-0c41-446f-ae06-443617ed99d9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:36:09.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=975e77a2-0271-4f0e-a201-1a89dc678b9a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:36:09.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=975e77a2-0271-4f0e-a201-1a89dc678b9a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@15ab9b71 2022-04-26T06:36:09.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=975e77a2-0271-4f0e-a201-1a89dc678b9a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:36:09.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=975e77a2-0271-4f0e-a201-1a89dc678b9a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:36:10.793Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=975e77a2-0271-4f0e-a201-1a89dc678b9a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:36:10.793Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=975e77a2-0271-4f0e-a201-1a89dc678b9a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:36:12.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a3ce95a-719f-4653-9d40-8a7e34bdddc3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:36:12.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a3ce95a-719f-4653-9d40-8a7e34bdddc3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@33948874 2022-04-26T06:36:12.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a3ce95a-719f-4653-9d40-8a7e34bdddc3 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:36:12.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a3ce95a-719f-4653-9d40-8a7e34bdddc3 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:36:13.837Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=7a3ce95a-719f-4653-9d40-8a7e34bdddc3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:36:13.838Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=7a3ce95a-719f-4653-9d40-8a7e34bdddc3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:36:15.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c7723d9-4ce1-4af9-b38c-80993e371698 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:36:15.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c7723d9-4ce1-4af9-b38c-80993e371698 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7bdb7dfa 2022-04-26T06:36:15.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c7723d9-4ce1-4af9-b38c-80993e371698 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:36:15.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c7723d9-4ce1-4af9-b38c-80993e371698 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:36:16.813Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3c7723d9-4ce1-4af9-b38c-80993e371698 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:36:16.813Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3c7723d9-4ce1-4af9-b38c-80993e371698 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:36:18.792Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aaca926b-d819-448a-9181-dcb19c1aebbb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:36:18.792Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aaca926b-d819-448a-9181-dcb19c1aebbb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3d0859e9 2022-04-26T06:36:18.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aaca926b-d819-448a-9181-dcb19c1aebbb PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:36:18.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aaca926b-d819-448a-9181-dcb19c1aebbb PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:36:19.853Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=aaca926b-d819-448a-9181-dcb19c1aebbb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:36:19.854Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=aaca926b-d819-448a-9181-dcb19c1aebbb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:36:20.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:36:20.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:36:21.245Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:35:14.376Z InvocationID=5a7d4e26-0c41-446f-ae06-443617ed99d9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:36:22.440Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a4063af-4505-4944-91d2-2f555e8f814b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:36:22.441Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a4063af-4505-4944-91d2-2f555e8f814b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3f1a5812 2022-04-26T06:36:22.441Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a4063af-4505-4944-91d2-2f555e8f814b PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:36:22.441Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a4063af-4505-4944-91d2-2f555e8f814b PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:36:23.465Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=2a4063af-4505-4944-91d2-2f555e8f814b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:36:23.466Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=2a4063af-4505-4944-91d2-2f555e8f814b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:36:25.431Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c727f578-2687-40fb-8b54-33dc03b2c64a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:36:25.431Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c727f578-2687-40fb-8b54-33dc03b2c64a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f9a17a2 2022-04-26T06:36:25.431Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c727f578-2687-40fb-8b54-33dc03b2c64a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:36:25.431Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c727f578-2687-40fb-8b54-33dc03b2c64a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:36:26.476Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c727f578-2687-40fb-8b54-33dc03b2c64a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:36:26.477Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c727f578-2687-40fb-8b54-33dc03b2c64a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:36:28.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2512844b-1624-4c17-8a4e-3edc90bc65c6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:36:28.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2512844b-1624-4c17-8a4e-3edc90bc65c6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2b5f7100 2022-04-26T06:36:28.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2512844b-1624-4c17-8a4e-3edc90bc65c6 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:36:28.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2512844b-1624-4c17-8a4e-3edc90bc65c6 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:36:29.448Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=2512844b-1624-4c17-8a4e-3edc90bc65c6 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:36:29.449Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=2512844b-1624-4c17-8a4e-3edc90bc65c6 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:36:31.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ff51289-d6a7-4e4b-86d8-1d874591483d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:36:31.440Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ff51289-d6a7-4e4b-86d8-1d874591483d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@12b91a07 2022-04-26T06:36:31.440Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ff51289-d6a7-4e4b-86d8-1d874591483d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:36:31.440Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ff51289-d6a7-4e4b-86d8-1d874591483d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:36:32.461Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=1ff51289-d6a7-4e4b-86d8-1d874591483d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:36:32.461Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=1ff51289-d6a7-4e4b-86d8-1d874591483d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:36:34.347Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=131d12c6-8e5a-46ba-a98f-9893ec346c22 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:36:34.347Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=131d12c6-8e5a-46ba-a98f-9893ec346c22 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@478bdf84 2022-04-26T06:36:34.347Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=131d12c6-8e5a-46ba-a98f-9893ec346c22 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:36:34.348Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=131d12c6-8e5a-46ba-a98f-9893ec346c22 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:36:35.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:36:35.369Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=131d12c6-8e5a-46ba-a98f-9893ec346c22 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:36:35.369Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=131d12c6-8e5a-46ba-a98f-9893ec346c22 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:36:35.383Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:36:37.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=23878783-ba5c-4509-8616-e7ba735be107 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:36:37.435Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=23878783-ba5c-4509-8616-e7ba735be107 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7985866f 2022-04-26T06:36:37.435Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=23878783-ba5c-4509-8616-e7ba735be107 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:36:37.435Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=23878783-ba5c-4509-8616-e7ba735be107 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:36:38.477Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=23878783-ba5c-4509-8616-e7ba735be107 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:36:38.477Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=23878783-ba5c-4509-8616-e7ba735be107 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:36:39.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea08a28f-1c55-4daf-bb9a-5b71589e689a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:36:39.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea08a28f-1c55-4daf-bb9a-5b71589e689a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@14cb8f5f 2022-04-26T06:36:39.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea08a28f-1c55-4daf-bb9a-5b71589e689a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:36:39.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea08a28f-1c55-4daf-bb9a-5b71589e689a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:36:40.813Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ea08a28f-1c55-4daf-bb9a-5b71589e689a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:36:40.814Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ea08a28f-1c55-4daf-bb9a-5b71589e689a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:36:42.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5248b87c-df9e-4f1f-b6be-e35c129a0d11 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:36:42.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5248b87c-df9e-4f1f-b6be-e35c129a0d11 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@544203f6 2022-04-26T06:36:42.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5248b87c-df9e-4f1f-b6be-e35c129a0d11 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:36:42.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5248b87c-df9e-4f1f-b6be-e35c129a0d11 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:36:43.785Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=5248b87c-df9e-4f1f-b6be-e35c129a0d11 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:36:43.786Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=5248b87c-df9e-4f1f-b6be-e35c129a0d11 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:36:45.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b88831b-d032-43ee-b9b2-96b33ea46a13 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:36:45.785Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b88831b-d032-43ee-b9b2-96b33ea46a13 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@390fd6fe 2022-04-26T06:36:45.785Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b88831b-d032-43ee-b9b2-96b33ea46a13 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:36:45.786Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b88831b-d032-43ee-b9b2-96b33ea46a13 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:36:46.829Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=7b88831b-d032-43ee-b9b2-96b33ea46a13 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:36:46.830Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=7b88831b-d032-43ee-b9b2-96b33ea46a13 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:36:46.853Z [qtp2085002312-17] ERROR o.j.g.database.StandardJanusGraph RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:35:14.376Z InvocationID=5a7d4e26-0c41-446f-ae06-443617ed99d9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Could not commit transaction [365] due to storage exception in commit org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT1M40S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 103 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 104 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:36:46.854Z [qtp2085002312-17] ERROR o.j.g.database.StandardJanusGraph RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:35:14.376Z InvocationID=5a7d4e26-0c41-446f-ae06-443617ed99d9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Could not commit transaction [365] due to exception org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT1M40S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 103 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 104 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:36:47.407Z [qtp2085002312-17] WARN o.o.sdc.be.servlets.ConsumerServlet RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:35:14.376Z InvocationID=5a7d4e26-0c41-446f-ae06-443617ed99d9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb #createConsumer('{"consumerName": "sdnc", "consumerSalt": "9cd4c3ad2a6f6ce3f3414e68b5157e63", "consumerPassword": "35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495"}', 'Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/consumers)@6d0411d6', 'jh0003'): org.apache.cxf.jaxrs.impl.ResponseImpl@fb5c443 in 1.55min (too slow!) 2022-04-26T06:36:47.408Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=93031 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:35:14.376Z InvocationID=5a7d4e26-0c41-446f-ae06-443617ed99d9 LogTimestamp=2022-04-26T06:36:47.407870Z ResponseDesc=Created Exiting. 2022-04-26T06:36:47.408Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:36:47.414Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=89542 PartnerName=kube-probe/1.22 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=ERROR ServerIPAddress=10.233.65.41 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2022-04-26T06:35:17.872Z InvocationID=5c99d6e3-be49-4b7c-803b-bf7214ba7fe0 ErrorCode=500 LogTimestamp=2022-04-26T06:36:47.414597Z ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:36:47.415Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservid HTTP/1.1 SC="500" 2022-04-26T06:36:47.419Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=36cb29a2-351e-4fc7-9b99-addd55da9acc InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2022-04-26T06:36:47.419712Z InvocationID=ac4f08b2-1b12-4d3c-8529-af098ea9588a LogTimestamp=2022-04-26T06:36:47.419780Z Entering 2022-04-26T06:36:47.420Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=36cb29a2-351e-4fc7-9b99-addd55da9acc InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2022-04-26T06:36:47.419712Z InvocationID=ac4f08b2-1b12-4d3c-8529-af098ea9588a LogTimestamp=2022-04-26T06:36:47.419780Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:36:47.422Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:36:47.420Z InvocationID=8eaeb618-5135-41a3-acb6-b8b530b9c115 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id vid 2022-04-26T06:36:47.954Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:36:47.420Z InvocationID=8eaeb618-5135-41a3-acb6-b8b530b9c115 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:36:48.012Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:36:47.420Z InvocationID=8eaeb618-5135-41a3-acb6-b8b530b9c115 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:36:48.071Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:36:47.420Z InvocationID=8eaeb618-5135-41a3-acb6-b8b530b9c115 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:36:48.144Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:36:47.420Z InvocationID=8eaeb618-5135-41a3-acb6-b8b530b9c115 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:36:48.207Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:36:47.420Z InvocationID=8eaeb618-5135-41a3-acb6-b8b530b9c115 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:36:48.264Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:36:47.420Z InvocationID=8eaeb618-5135-41a3-acb6-b8b530b9c115 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:36:48.371Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:36:47.420Z InvocationID=8eaeb618-5135-41a3-acb6-b8b530b9c115 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:36:48.477Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:36:47.420Z InvocationID=8eaeb618-5135-41a3-acb6-b8b530b9c115 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:36:48.584Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:36:47.420Z InvocationID=8eaeb618-5135-41a3-acb6-b8b530b9c115 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:36:48.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4029092-b00a-49b3-9704-55f0c7e97d2c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:36:48.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4029092-b00a-49b3-9704-55f0c7e97d2c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@76e611c5 2022-04-26T06:36:48.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4029092-b00a-49b3-9704-55f0c7e97d2c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:36:48.785Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4029092-b00a-49b3-9704-55f0c7e97d2c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:36:48.827Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:36:47.420Z InvocationID=8eaeb618-5135-41a3-acb6-b8b530b9c115 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:36:49.234Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:36:47.420Z InvocationID=8eaeb618-5135-41a3-acb6-b8b530b9c115 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:36:49.801Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e4029092-b00a-49b3-9704-55f0c7e97d2c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:36:49.801Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e4029092-b00a-49b3-9704-55f0c7e97d2c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:36:50.040Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:36:47.420Z InvocationID=8eaeb618-5135-41a3-acb6-b8b530b9c115 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:36:50.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:36:50.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:36:50.928Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:36:47.420Z InvocationID=8eaeb618-5135-41a3-acb6-b8b530b9c115 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:36:52.537Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:36:47.420Z InvocationID=8eaeb618-5135-41a3-acb6-b8b530b9c115 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:36:52.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb845a3c-3ff0-4558-9383-f69a866a2e21 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:36:52.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb845a3c-3ff0-4558-9383-f69a866a2e21 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@70594803 2022-04-26T06:36:52.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb845a3c-3ff0-4558-9383-f69a866a2e21 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:36:52.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb845a3c-3ff0-4558-9383-f69a866a2e21 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:36:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:36:52.686Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:36:52.686Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:36:53.645Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=cb845a3c-3ff0-4558-9383-f69a866a2e21 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:36:53.645Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=cb845a3c-3ff0-4558-9383-f69a866a2e21 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:36:54.228Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:36:47.420Z InvocationID=8eaeb618-5135-41a3-acb6-b8b530b9c115 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:36:55.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eb7b0dc3-ff5d-43b2-a625-9335b8cfc96a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:36:55.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eb7b0dc3-ff5d-43b2-a625-9335b8cfc96a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7b8b301e 2022-04-26T06:36:55.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eb7b0dc3-ff5d-43b2-a625-9335b8cfc96a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:36:55.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eb7b0dc3-ff5d-43b2-a625-9335b8cfc96a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:36:56.461Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=eb7b0dc3-ff5d-43b2-a625-9335b8cfc96a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:36:56.461Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=eb7b0dc3-ff5d-43b2-a625-9335b8cfc96a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:36:57.434Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:36:47.420Z InvocationID=8eaeb618-5135-41a3-acb6-b8b530b9c115 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:36:58.526Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0130f3e-61cb-4e20-996c-df73dd43cb83 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:36:58.526Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0130f3e-61cb-4e20-996c-df73dd43cb83 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@408e76fe 2022-04-26T06:36:58.526Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0130f3e-61cb-4e20-996c-df73dd43cb83 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:36:58.526Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0130f3e-61cb-4e20-996c-df73dd43cb83 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:36:59.561Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f0130f3e-61cb-4e20-996c-df73dd43cb83 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:36:59.562Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f0130f3e-61cb-4e20-996c-df73dd43cb83 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:37:01.334Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48ae720f-7083-4b87-b2fc-63127c5990bf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:37:01.335Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48ae720f-7083-4b87-b2fc-63127c5990bf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7426fef0 2022-04-26T06:37:01.335Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48ae720f-7083-4b87-b2fc-63127c5990bf PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:37:01.335Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48ae720f-7083-4b87-b2fc-63127c5990bf PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:37:02.376Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=48ae720f-7083-4b87-b2fc-63127c5990bf ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:37:02.377Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=48ae720f-7083-4b87-b2fc-63127c5990bf ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:37:03.944Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:36:47.420Z InvocationID=8eaeb618-5135-41a3-acb6-b8b530b9c115 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:04.442Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60f2ab5c-aca8-47e5-9384-4633ee654788 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:37:04.442Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60f2ab5c-aca8-47e5-9384-4633ee654788 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@541599e7 2022-04-26T06:37:04.444Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60f2ab5c-aca8-47e5-9384-4633ee654788 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:37:04.444Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60f2ab5c-aca8-47e5-9384-4633ee654788 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:37:05.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:37:05.374Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:37:05.497Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=60f2ab5c-aca8-47e5-9384-4633ee654788 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:37:05.498Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=60f2ab5c-aca8-47e5-9384-4633ee654788 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:37:07.331Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=210df96c-8662-4c35-a60c-8225a9bb60bd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:37:07.331Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=210df96c-8662-4c35-a60c-8225a9bb60bd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@385884ba 2022-04-26T06:37:07.331Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=210df96c-8662-4c35-a60c-8225a9bb60bd PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:37:07.331Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=210df96c-8662-4c35-a60c-8225a9bb60bd PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:37:07.645Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:37:08.360Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=210df96c-8662-4c35-a60c-8225a9bb60bd ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:37:08.361Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=210df96c-8662-4c35-a60c-8225a9bb60bd ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:37:09.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f77b95db-853e-4ba2-bceb-bd337e4fed96 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:37:09.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f77b95db-853e-4ba2-bceb-bd337e4fed96 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5e28ea80 2022-04-26T06:37:09.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f77b95db-853e-4ba2-bceb-bd337e4fed96 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:37:09.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f77b95db-853e-4ba2-bceb-bd337e4fed96 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:37:10.799Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f77b95db-853e-4ba2-bceb-bd337e4fed96 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:37:10.799Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f77b95db-853e-4ba2-bceb-bd337e4fed96 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:37:12.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90d50a82-60fb-44fb-9cd9-c09f0922e9f5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:37:12.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90d50a82-60fb-44fb-9cd9-c09f0922e9f5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8ebe2a7 2022-04-26T06:37:12.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90d50a82-60fb-44fb-9cd9-c09f0922e9f5 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:37:12.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90d50a82-60fb-44fb-9cd9-c09f0922e9f5 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:37:13.800Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=90d50a82-60fb-44fb-9cd9-c09f0922e9f5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:37:13.801Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=90d50a82-60fb-44fb-9cd9-c09f0922e9f5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:37:15.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=756ab96e-fa71-4f06-880c-14ee8a05e37f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:37:15.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=756ab96e-fa71-4f06-880c-14ee8a05e37f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@63de75bd 2022-04-26T06:37:15.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=756ab96e-fa71-4f06-880c-14ee8a05e37f PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:37:15.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=756ab96e-fa71-4f06-880c-14ee8a05e37f PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:37:16.751Z [qtp2085002312-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:36:47.420Z InvocationID=8eaeb618-5135-41a3-acb6-b8b530b9c115 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:16.809Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=756ab96e-fa71-4f06-880c-14ee8a05e37f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:37:16.809Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=756ab96e-fa71-4f06-880c-14ee8a05e37f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:37:18.794Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df73a20f-b261-43aa-a3e8-f9d8d6744ee5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:37:18.794Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df73a20f-b261-43aa-a3e8-f9d8d6744ee5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@161174bd 2022-04-26T06:37:18.795Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df73a20f-b261-43aa-a3e8-f9d8d6744ee5 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:37:18.795Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df73a20f-b261-43aa-a3e8-f9d8d6744ee5 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:37:19.817Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=df73a20f-b261-43aa-a3e8-f9d8d6744ee5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:37:19.818Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=df73a20f-b261-43aa-a3e8-f9d8d6744ee5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:37:20.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:37:20.379Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:37:22.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e607211f-9800-4324-bd92-5829558922f2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:37:22.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e607211f-9800-4324-bd92-5829558922f2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2388bd5e 2022-04-26T06:37:22.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e607211f-9800-4324-bd92-5829558922f2 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:37:22.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e607211f-9800-4324-bd92-5829558922f2 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:37:23.465Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e607211f-9800-4324-bd92-5829558922f2 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:37:23.466Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e607211f-9800-4324-bd92-5829558922f2 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:37:25.426Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=362f9bdd-a60e-476a-90fb-7ab55ab8acb4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:37:25.427Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=362f9bdd-a60e-476a-90fb-7ab55ab8acb4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3ddd9a07 2022-04-26T06:37:25.427Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=362f9bdd-a60e-476a-90fb-7ab55ab8acb4 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:37:25.427Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=362f9bdd-a60e-476a-90fb-7ab55ab8acb4 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:37:26.441Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=362f9bdd-a60e-476a-90fb-7ab55ab8acb4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:37:26.441Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=362f9bdd-a60e-476a-90fb-7ab55ab8acb4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:37:28.526Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6426a8d-d47b-4a29-9668-afd6f6f86a11 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:37:28.527Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6426a8d-d47b-4a29-9668-afd6f6f86a11 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@32f3b6ff 2022-04-26T06:37:28.527Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6426a8d-d47b-4a29-9668-afd6f6f86a11 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:37:28.528Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6426a8d-d47b-4a29-9668-afd6f6f86a11 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:37:29.550Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e6426a8d-d47b-4a29-9668-afd6f6f86a11 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:37:29.550Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e6426a8d-d47b-4a29-9668-afd6f6f86a11 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:37:31.343Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f56dba3d-4f2a-4a72-914b-ba98f6118b07 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:37:31.343Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f56dba3d-4f2a-4a72-914b-ba98f6118b07 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@66efba3c 2022-04-26T06:37:31.343Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f56dba3d-4f2a-4a72-914b-ba98f6118b07 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:37:31.343Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f56dba3d-4f2a-4a72-914b-ba98f6118b07 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:37:32.364Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f56dba3d-4f2a-4a72-914b-ba98f6118b07 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:37:32.364Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f56dba3d-4f2a-4a72-914b-ba98f6118b07 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:37:34.442Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73c76b21-c295-49aa-9042-6b435e8b8341 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:37:34.442Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73c76b21-c295-49aa-9042-6b435e8b8341 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2100427c 2022-04-26T06:37:34.442Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73c76b21-c295-49aa-9042-6b435e8b8341 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:37:34.443Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73c76b21-c295-49aa-9042-6b435e8b8341 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:37:35.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:37:35.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:37:35.465Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=73c76b21-c295-49aa-9042-6b435e8b8341 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:37:35.465Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=73c76b21-c295-49aa-9042-6b435e8b8341 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:37:37.333Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f37ef6a9-bf75-42e9-be75-c4f93181b5fb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:37:37.333Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f37ef6a9-bf75-42e9-be75-c4f93181b5fb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@15a270ee 2022-04-26T06:37:37.333Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f37ef6a9-bf75-42e9-be75-c4f93181b5fb PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:37:37.333Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f37ef6a9-bf75-42e9-be75-c4f93181b5fb PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:37:38.345Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f37ef6a9-bf75-42e9-be75-c4f93181b5fb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:37:38.345Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f37ef6a9-bf75-42e9-be75-c4f93181b5fb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:37:39.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90d90841-0810-459f-bda3-f797ee028698 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:37:39.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90d90841-0810-459f-bda3-f797ee028698 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@62e5a177 2022-04-26T06:37:39.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90d90841-0810-459f-bda3-f797ee028698 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:37:39.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90d90841-0810-459f-bda3-f797ee028698 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:37:40.812Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=90d90841-0810-459f-bda3-f797ee028698 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:37:40.813Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=90d90841-0810-459f-bda3-f797ee028698 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:37:42.358Z [qtp2085002312-16] ERROR o.j.g.database.StandardJanusGraph RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:36:47.420Z InvocationID=8eaeb618-5135-41a3-acb6-b8b530b9c115 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Could not commit transaction [368] due to storage exception in commit org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT1M40S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 103 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 104 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:42.358Z [qtp2085002312-16] ERROR o.j.g.database.StandardJanusGraph RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:36:47.420Z InvocationID=8eaeb618-5135-41a3-acb6-b8b530b9c115 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Could not commit transaction [368] due to exception org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT1M40S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 103 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 104 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:42.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0e4cac7-a002-4360-9afb-d636b5f09255 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:37:42.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0e4cac7-a002-4360-9afb-d636b5f09255 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@52dd7ad7 2022-04-26T06:37:42.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0e4cac7-a002-4360-9afb-d636b5f09255 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:37:42.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0e4cac7-a002-4360-9afb-d636b5f09255 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:37:42.902Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=55482 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:36:47.420Z InvocationID=8eaeb618-5135-41a3-acb6-b8b530b9c115 LogTimestamp=2022-04-26T06:37:42.902567Z ResponseDesc=Created Exiting. 2022-04-26T06:37:42.903Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:37:42.909Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=25038 PartnerName=kube-probe/1.22 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=ERROR ServerIPAddress=10.233.65.41 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2022-04-26T06:37:17.871Z InvocationID=ab46b65f-22e5-4b8a-b94d-94ed80e499c0 ErrorCode=500 LogTimestamp=2022-04-26T06:37:42.909516Z ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:37:42.910Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercognita HTTP/1.1 SC="500" 2022-04-26T06:37:42.916Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=bc9a5d3e-4300-459a-806c-ad558057a495 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2022-04-26T06:37:42.916201Z InvocationID=d23bec58-6231-485c-b37d-eec4723d2127 LogTimestamp=2022-04-26T06:37:42.916285Z Entering 2022-04-26T06:37:42.916Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=bc9a5d3e-4300-459a-806c-ad558057a495 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2022-04-26T06:37:42.916201Z InvocationID=d23bec58-6231-485c-b37d-eec4723d2127 LogTimestamp=2022-04-26T06:37:42.916285Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:37:42.919Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id cognita 2022-04-26T06:37:43.448Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:43.481Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:43.513Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:43.545Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:43.576Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:43.608Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:43.665Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:43.722Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:43.821Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=b0e4cac7-a002-4360-9afb-d636b5f09255 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:37:43.822Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=b0e4cac7-a002-4360-9afb-d636b5f09255 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:37:43.829Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:43.936Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:44.043Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:44.249Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:44.657Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:45.063Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:45.470Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:45.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6ccd2f6-c689-41d1-b90b-2fb467fbc30c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:37:45.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6ccd2f6-c689-41d1-b90b-2fb467fbc30c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@74840115 2022-04-26T06:37:45.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6ccd2f6-c689-41d1-b90b-2fb467fbc30c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:37:45.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6ccd2f6-c689-41d1-b90b-2fb467fbc30c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:37:45.877Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:46.684Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:46.797Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f6ccd2f6-c689-41d1-b90b-2fb467fbc30c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:37:46.797Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f6ccd2f6-c689-41d1-b90b-2fb467fbc30c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:37:47.491Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:48.300Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:48.786Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e491faa-ca86-420f-b881-ffcee5a2f476 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:37:48.786Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e491faa-ca86-420f-b881-ffcee5a2f476 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6b436a0d 2022-04-26T06:37:48.786Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e491faa-ca86-420f-b881-ffcee5a2f476 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:37:48.786Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e491faa-ca86-420f-b881-ffcee5a2f476 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:37:49.801Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=5e491faa-ca86-420f-b881-ffcee5a2f476 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:37:49.802Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=5e491faa-ca86-420f-b881-ffcee5a2f476 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:37:49.931Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:50.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:37:50.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:37:51.538Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:52.431Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37a21654-71ba-4501-9fb4-c33082fbb94a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:37:52.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37a21654-71ba-4501-9fb4-c33082fbb94a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2b22e9b0 2022-04-26T06:37:52.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37a21654-71ba-4501-9fb4-c33082fbb94a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:37:52.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37a21654-71ba-4501-9fb4-c33082fbb94a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:37:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:37:52.686Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:37:52.686Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:37:53.225Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:53.449Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=37a21654-71ba-4501-9fb4-c33082fbb94a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:37:53.450Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=37a21654-71ba-4501-9fb4-c33082fbb94a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:37:55.426Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31a4d45d-c1f6-479a-bac1-5096d65da11b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:37:55.426Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31a4d45d-c1f6-479a-bac1-5096d65da11b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@425411c 2022-04-26T06:37:55.426Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31a4d45d-c1f6-479a-bac1-5096d65da11b PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:37:55.427Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31a4d45d-c1f6-479a-bac1-5096d65da11b PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:37:56.435Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:37:56.457Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=31a4d45d-c1f6-479a-bac1-5096d65da11b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:37:56.458Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=31a4d45d-c1f6-479a-bac1-5096d65da11b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:37:58.426Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=415c04cb-2609-4b33-9cb2-87ffd7560cc0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:37:58.427Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=415c04cb-2609-4b33-9cb2-87ffd7560cc0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@11ba6cd4 2022-04-26T06:37:58.428Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=415c04cb-2609-4b33-9cb2-87ffd7560cc0 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:37:58.428Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=415c04cb-2609-4b33-9cb2-87ffd7560cc0 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:37:59.465Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=415c04cb-2609-4b33-9cb2-87ffd7560cc0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:37:59.465Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=415c04cb-2609-4b33-9cb2-87ffd7560cc0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:37:59.734Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:38:01.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79d5f5e8-bf1f-41a2-9e4e-b057ba8db19b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:38:01.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79d5f5e8-bf1f-41a2-9e4e-b057ba8db19b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@794ddc2a 2022-04-26T06:38:01.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79d5f5e8-bf1f-41a2-9e4e-b057ba8db19b PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:38:01.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79d5f5e8-bf1f-41a2-9e4e-b057ba8db19b PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:38:02.472Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=79d5f5e8-bf1f-41a2-9e4e-b057ba8db19b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:38:02.473Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=79d5f5e8-bf1f-41a2-9e4e-b057ba8db19b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:38:02.941Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:38:04.345Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f40c7dd4-8564-4322-ab8a-2e5a16386aa0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:38:04.345Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f40c7dd4-8564-4322-ab8a-2e5a16386aa0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5d213ba3 2022-04-26T06:38:04.345Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f40c7dd4-8564-4322-ab8a-2e5a16386aa0 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:38:04.346Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f40c7dd4-8564-4322-ab8a-2e5a16386aa0 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:38:05.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:38:05.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:38:05.384Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f40c7dd4-8564-4322-ab8a-2e5a16386aa0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:38:05.385Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f40c7dd4-8564-4322-ab8a-2e5a16386aa0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:38:07.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=588f36c7-47e3-466d-800c-9d3672f2d3e1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:38:07.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=588f36c7-47e3-466d-800c-9d3672f2d3e1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6fa2a8f6 2022-04-26T06:38:07.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=588f36c7-47e3-466d-800c-9d3672f2d3e1 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:38:07.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=588f36c7-47e3-466d-800c-9d3672f2d3e1 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:38:07.664Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:38:08.457Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=588f36c7-47e3-466d-800c-9d3672f2d3e1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:38:08.457Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=588f36c7-47e3-466d-800c-9d3672f2d3e1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:38:09.347Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:38:09.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ecf53b6-993a-4c89-a182-537b1d3cedba PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:38:09.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ecf53b6-993a-4c89-a182-537b1d3cedba PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5d915ea1 2022-04-26T06:38:09.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ecf53b6-993a-4c89-a182-537b1d3cedba PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:38:09.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ecf53b6-993a-4c89-a182-537b1d3cedba PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:38:10.793Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=9ecf53b6-993a-4c89-a182-537b1d3cedba ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:38:10.793Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=9ecf53b6-993a-4c89-a182-537b1d3cedba ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:38:12.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15d38289-96c5-4e98-89c3-f3b7a3074a58 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:38:12.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15d38289-96c5-4e98-89c3-f3b7a3074a58 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@13998991 2022-04-26T06:38:12.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15d38289-96c5-4e98-89c3-f3b7a3074a58 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:38:12.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15d38289-96c5-4e98-89c3-f3b7a3074a58 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:38:13.805Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=15d38289-96c5-4e98-89c3-f3b7a3074a58 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:38:13.806Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=15d38289-96c5-4e98-89c3-f3b7a3074a58 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:38:15.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68f17613-771f-44fa-8f28-d1b47df60a91 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:38:15.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68f17613-771f-44fa-8f28-d1b47df60a91 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1d62aef 2022-04-26T06:38:15.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68f17613-771f-44fa-8f28-d1b47df60a91 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:38:15.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68f17613-771f-44fa-8f28-d1b47df60a91 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:38:16.813Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=68f17613-771f-44fa-8f28-d1b47df60a91 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:38:16.813Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=68f17613-771f-44fa-8f28-d1b47df60a91 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:38:18.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32002b1e-369f-434f-91ec-98fe89f43148 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:38:18.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32002b1e-369f-434f-91ec-98fe89f43148 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@44ab8f3 2022-04-26T06:38:18.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32002b1e-369f-434f-91ec-98fe89f43148 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:38:18.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32002b1e-369f-434f-91ec-98fe89f43148 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:38:19.795Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=32002b1e-369f-434f-91ec-98fe89f43148 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:38:19.795Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=32002b1e-369f-434f-91ec-98fe89f43148 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:38:20.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:38:20.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:38:22.225Z [qtp2085002312-13] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:38:22.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ed710e3-8bb6-4a8d-8e29-85e4ee232358 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:38:22.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ed710e3-8bb6-4a8d-8e29-85e4ee232358 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4af6a569 2022-04-26T06:38:22.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ed710e3-8bb6-4a8d-8e29-85e4ee232358 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:38:22.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ed710e3-8bb6-4a8d-8e29-85e4ee232358 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:38:23.663Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=6ed710e3-8bb6-4a8d-8e29-85e4ee232358 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:38:23.664Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=6ed710e3-8bb6-4a8d-8e29-85e4ee232358 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:38:25.440Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3fcd8512-c407-47cf-bb6c-d40309b27101 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:38:25.440Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3fcd8512-c407-47cf-bb6c-d40309b27101 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@430ac49a 2022-04-26T06:38:25.440Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3fcd8512-c407-47cf-bb6c-d40309b27101 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:38:25.440Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3fcd8512-c407-47cf-bb6c-d40309b27101 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:38:26.477Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3fcd8512-c407-47cf-bb6c-d40309b27101 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:38:26.478Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3fcd8512-c407-47cf-bb6c-d40309b27101 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:38:28.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6fc095e-cef2-4c0c-844d-2a9c02ea63aa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:38:28.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6fc095e-cef2-4c0c-844d-2a9c02ea63aa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7d626e32 2022-04-26T06:38:28.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6fc095e-cef2-4c0c-844d-2a9c02ea63aa PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:38:28.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6fc095e-cef2-4c0c-844d-2a9c02ea63aa PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:38:29.645Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c6fc095e-cef2-4c0c-844d-2a9c02ea63aa ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:38:29.647Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c6fc095e-cef2-4c0c-844d-2a9c02ea63aa ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:38:31.540Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7127e0df-6f69-46c4-a690-1df44235f948 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:38:31.541Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7127e0df-6f69-46c4-a690-1df44235f948 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@49763d79 2022-04-26T06:38:31.541Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7127e0df-6f69-46c4-a690-1df44235f948 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:38:31.541Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7127e0df-6f69-46c4-a690-1df44235f948 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:38:32.585Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=7127e0df-6f69-46c4-a690-1df44235f948 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:38:32.585Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=7127e0df-6f69-46c4-a690-1df44235f948 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:38:34.433Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a2df1c8-b1d0-4c4a-9ba8-63a52e926c38 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:38:34.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a2df1c8-b1d0-4c4a-9ba8-63a52e926c38 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@27bc4963 2022-04-26T06:38:34.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a2df1c8-b1d0-4c4a-9ba8-63a52e926c38 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:38:34.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a2df1c8-b1d0-4c4a-9ba8-63a52e926c38 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:38:35.368Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:38:35.381Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:38:35.465Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=2a2df1c8-b1d0-4c4a-9ba8-63a52e926c38 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:38:35.465Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=2a2df1c8-b1d0-4c4a-9ba8-63a52e926c38 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:38:37.338Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e363c59a-8702-4d6c-856c-b1854cd67329 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:38:37.338Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e363c59a-8702-4d6c-856c-b1854cd67329 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6bc3f5ee 2022-04-26T06:38:37.338Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e363c59a-8702-4d6c-856c-b1854cd67329 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:38:37.338Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e363c59a-8702-4d6c-856c-b1854cd67329 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:38:38.377Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e363c59a-8702-4d6c-856c-b1854cd67329 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:38:38.378Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e363c59a-8702-4d6c-856c-b1854cd67329 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:38:39.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5eef082f-82a8-4fc6-bfff-e8886a0f00cd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:38:39.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5eef082f-82a8-4fc6-bfff-e8886a0f00cd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6a986353 2022-04-26T06:38:39.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5eef082f-82a8-4fc6-bfff-e8886a0f00cd PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:38:39.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5eef082f-82a8-4fc6-bfff-e8886a0f00cd PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:38:40.777Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=5eef082f-82a8-4fc6-bfff-e8886a0f00cd ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:38:40.777Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=5eef082f-82a8-4fc6-bfff-e8886a0f00cd ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:38:42.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f314ef57-4597-44ca-bc28-21185dd8a4ac PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:38:42.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f314ef57-4597-44ca-bc28-21185dd8a4ac PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3979ffee 2022-04-26T06:38:42.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f314ef57-4597-44ca-bc28-21185dd8a4ac PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:38:42.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f314ef57-4597-44ca-bc28-21185dd8a4ac PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:38:43.786Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f314ef57-4597-44ca-bc28-21185dd8a4ac ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:38:43.786Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f314ef57-4597-44ca-bc28-21185dd8a4ac ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:38:45.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1ebe9bd-456c-4553-8825-1a80b95dfc01 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:38:45.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1ebe9bd-456c-4553-8825-1a80b95dfc01 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@69054788 2022-04-26T06:38:45.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1ebe9bd-456c-4553-8825-1a80b95dfc01 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:38:45.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1ebe9bd-456c-4553-8825-1a80b95dfc01 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:38:46.801Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f1ebe9bd-456c-4553-8825-1a80b95dfc01 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:38:46.801Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f1ebe9bd-456c-4553-8825-1a80b95dfc01 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:38:47.835Z [qtp2085002312-13] ERROR o.j.g.database.StandardJanusGraph RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Could not commit transaction [371] due to storage exception in commit org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT1M40S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 103 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 104 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:38:47.837Z [qtp2085002312-13] ERROR o.j.g.database.StandardJanusGraph RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Could not commit transaction [371] due to exception org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT1M40S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 103 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 104 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:38:48.378Z [qtp2085002312-13] WARN o.o.sdc.be.servlets.ConsumerServlet RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb #createConsumer('{"consumerName": "cognita", "consumerSalt": "9cd4c3ad2a6f6ce3f3414e68b5157e63", "consumerPassword": "35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495"}', 'Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/consumers)@6d0411d6', 'jh0003'): org.apache.cxf.jaxrs.impl.ResponseImpl@55c6bcd9 in 1.09min (too slow!) 2022-04-26T06:38:48.381Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=65463 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:37:42.917Z InvocationID=65b9e01e-38d0-4c26-bb89-7d4867b1d830 LogTimestamp=2022-04-26T06:38:48.380647Z ResponseDesc=Created Exiting. 2022-04-26T06:38:48.384Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:38:48.399Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=150527 PartnerName=kube-probe/1.22 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=ERROR ServerIPAddress=10.233.65.41 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2022-04-26T06:36:17.872Z InvocationID=d953bd2a-aa8b-484b-8266-c41b865eb389 ErrorCode=500 LogTimestamp=2022-04-26T06:38:48.399578Z ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:38:48.400Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userclamp HTTP/1.1 SC="500" 2022-04-26T06:38:48.406Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=45124756-afab-4534-a40c-66e113802b80 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:38:48.406251Z InvocationID=097a3716-086e-4426-9892-fca1ff2510d0 LogTimestamp=2022-04-26T06:38:48.406331Z Entering 2022-04-26T06:38:48.406Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=45124756-afab-4534-a40c-66e113802b80 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:38:48.406251Z InvocationID=097a3716-086e-4426-9892-fca1ff2510d0 LogTimestamp=2022-04-26T06:38:48.406331Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:38:48.413Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id clamp 2022-04-26T06:38:48.785Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57210b55-d4d4-4ada-8ec8-15af717c9834 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:38:48.786Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57210b55-d4d4-4ada-8ec8-15af717c9834 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5a2809bc 2022-04-26T06:38:48.786Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57210b55-d4d4-4ada-8ec8-15af717c9834 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:38:48.786Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57210b55-d4d4-4ada-8ec8-15af717c9834 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:38:48.938Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:38:48.995Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:38:49.057Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:38:49.115Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:38:49.173Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:38:49.229Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:38:49.287Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:38:49.394Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:38:49.601Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:38:49.801Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=57210b55-d4d4-4ada-8ec8-15af717c9834 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:38:49.801Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=57210b55-d4d4-4ada-8ec8-15af717c9834 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:38:50.007Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:38:50.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:38:50.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:38:50.424Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:38:50.830Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:38:51.236Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:38:52.124Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:38:52.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=121b0518-2508-4217-8401-e807f4a10995 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:38:52.531Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=121b0518-2508-4217-8401-e807f4a10995 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@287cb0bd 2022-04-26T06:38:52.531Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=121b0518-2508-4217-8401-e807f4a10995 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:38:52.531Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=121b0518-2508-4217-8401-e807f4a10995 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:38:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:38:52.687Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:38:52.687Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:38:52.931Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:38:53.544Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=121b0518-2508-4217-8401-e807f4a10995 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:38:53.545Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=121b0518-2508-4217-8401-e807f4a10995 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:38:53.737Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:38:54.624Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:38:55.436Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c949ef2e-d46a-4ca1-b1db-1c28e0b6d149 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:38:55.437Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c949ef2e-d46a-4ca1-b1db-1c28e0b6d149 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5e9d6688 2022-04-26T06:38:55.437Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c949ef2e-d46a-4ca1-b1db-1c28e0b6d149 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:38:55.437Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c949ef2e-d46a-4ca1-b1db-1c28e0b6d149 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:38:56.247Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:38:56.461Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c949ef2e-d46a-4ca1-b1db-1c28e0b6d149 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:38:56.461Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c949ef2e-d46a-4ca1-b1db-1c28e0b6d149 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:38:57.925Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:38:58.436Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0e96f59-e37f-434b-99ef-120992294216 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:38:58.437Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0e96f59-e37f-434b-99ef-120992294216 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4ceac3c4 2022-04-26T06:38:58.437Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0e96f59-e37f-434b-99ef-120992294216 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:38:58.437Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0e96f59-e37f-434b-99ef-120992294216 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:38:59.465Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e0e96f59-e37f-434b-99ef-120992294216 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:38:59.466Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e0e96f59-e37f-434b-99ef-120992294216 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:39:01.143Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:39:01.344Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=404f89ff-025e-4f37-820a-c80934b133dc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:39:01.346Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=404f89ff-025e-4f37-820a-c80934b133dc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2a555cb5 2022-04-26T06:39:01.346Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=404f89ff-025e-4f37-820a-c80934b133dc PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:39:01.346Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=404f89ff-025e-4f37-820a-c80934b133dc PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:39:02.377Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=404f89ff-025e-4f37-820a-c80934b133dc ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:39:02.378Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=404f89ff-025e-4f37-820a-c80934b133dc ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:39:04.430Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba72855a-2830-4ffa-b1a0-f95f35a7229c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:39:04.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba72855a-2830-4ffa-b1a0-f95f35a7229c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4207297d 2022-04-26T06:39:04.441Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba72855a-2830-4ffa-b1a0-f95f35a7229c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:39:04.441Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba72855a-2830-4ffa-b1a0-f95f35a7229c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:39:05.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:39:05.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:39:05.482Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ba72855a-2830-4ffa-b1a0-f95f35a7229c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:39:05.483Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ba72855a-2830-4ffa-b1a0-f95f35a7229c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:39:07.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40a92887-48a5-4002-b2cd-dc9fc0d85280 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:39:07.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40a92887-48a5-4002-b2cd-dc9fc0d85280 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@13a869ad 2022-04-26T06:39:07.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40a92887-48a5-4002-b2cd-dc9fc0d85280 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:39:07.435Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40a92887-48a5-4002-b2cd-dc9fc0d85280 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:39:07.633Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:39:07.683Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:39:08.462Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=40a92887-48a5-4002-b2cd-dc9fc0d85280 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:39:08.462Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=40a92887-48a5-4002-b2cd-dc9fc0d85280 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:39:09.785Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9934112d-8537-4c8f-a195-d26d02b78cc9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:39:09.785Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9934112d-8537-4c8f-a195-d26d02b78cc9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@794d28e1 2022-04-26T06:39:09.785Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9934112d-8537-4c8f-a195-d26d02b78cc9 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:39:09.786Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9934112d-8537-4c8f-a195-d26d02b78cc9 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:39:10.825Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=9934112d-8537-4c8f-a195-d26d02b78cc9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:39:10.825Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=9934112d-8537-4c8f-a195-d26d02b78cc9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:39:12.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=17e95be4-ec67-4291-941c-cc44ef21b732 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:39:12.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=17e95be4-ec67-4291-941c-cc44ef21b732 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7bce6553 2022-04-26T06:39:12.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=17e95be4-ec67-4291-941c-cc44ef21b732 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:39:12.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=17e95be4-ec67-4291-941c-cc44ef21b732 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:39:13.805Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=17e95be4-ec67-4291-941c-cc44ef21b732 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:39:13.806Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=17e95be4-ec67-4291-941c-cc44ef21b732 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:39:14.040Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:39:15.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66f3d4af-d9cd-4049-bbf1-6bf92bbfdfa8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:39:15.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66f3d4af-d9cd-4049-bbf1-6bf92bbfdfa8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3cd1b255 2022-04-26T06:39:15.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66f3d4af-d9cd-4049-bbf1-6bf92bbfdfa8 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:39:15.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66f3d4af-d9cd-4049-bbf1-6bf92bbfdfa8 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:39:16.810Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=66f3d4af-d9cd-4049-bbf1-6bf92bbfdfa8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:39:16.811Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=66f3d4af-d9cd-4049-bbf1-6bf92bbfdfa8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:39:18.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8c636e7-d447-4aea-91d8-f44d5102e44d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:39:18.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8c636e7-d447-4aea-91d8-f44d5102e44d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@43156e8a 2022-04-26T06:39:18.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8c636e7-d447-4aea-91d8-f44d5102e44d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:39:18.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8c636e7-d447-4aea-91d8-f44d5102e44d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:39:19.817Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c8c636e7-d447-4aea-91d8-f44d5102e44d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:39:19.817Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c8c636e7-d447-4aea-91d8-f44d5102e44d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:39:20.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:39:20.379Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:39:20.523Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:39:22.533Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61889130-db38-4a38-8479-9062ed9ce064 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:39:22.533Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61889130-db38-4a38-8479-9062ed9ce064 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2346fa7 2022-04-26T06:39:22.533Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61889130-db38-4a38-8479-9062ed9ce064 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:39:22.533Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61889130-db38-4a38-8479-9062ed9ce064 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:39:23.565Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=61889130-db38-4a38-8479-9062ed9ce064 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:39:23.565Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=61889130-db38-4a38-8479-9062ed9ce064 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:39:25.324Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a9c9153-e271-44b7-8999-c500de3d1f8c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:39:25.325Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a9c9153-e271-44b7-8999-c500de3d1f8c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6a368d7f 2022-04-26T06:39:25.325Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a9c9153-e271-44b7-8999-c500de3d1f8c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:39:25.325Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a9c9153-e271-44b7-8999-c500de3d1f8c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:39:26.348Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=2a9c9153-e271-44b7-8999-c500de3d1f8c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:39:26.349Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=2a9c9153-e271-44b7-8999-c500de3d1f8c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:39:28.529Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=321326a2-c04d-4ddc-94f8-6d5c4118eda9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:39:28.529Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=321326a2-c04d-4ddc-94f8-6d5c4118eda9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@491e61e7 2022-04-26T06:39:28.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=321326a2-c04d-4ddc-94f8-6d5c4118eda9 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:39:28.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=321326a2-c04d-4ddc-94f8-6d5c4118eda9 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:39:29.547Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=321326a2-c04d-4ddc-94f8-6d5c4118eda9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:39:29.550Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=321326a2-c04d-4ddc-94f8-6d5c4118eda9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:39:31.331Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb7280b6-a983-4eea-aa78-5a4ce11aa3e9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:39:31.331Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb7280b6-a983-4eea-aa78-5a4ce11aa3e9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6bca4f59 2022-04-26T06:39:31.331Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb7280b6-a983-4eea-aa78-5a4ce11aa3e9 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:39:31.331Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb7280b6-a983-4eea-aa78-5a4ce11aa3e9 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:39:32.361Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=fb7280b6-a983-4eea-aa78-5a4ce11aa3e9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:39:32.362Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=fb7280b6-a983-4eea-aa78-5a4ce11aa3e9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:39:33.333Z [qtp2085002312-20] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:39:34.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ae3c454-7598-4df3-9f8e-e8362af020a5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:39:34.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ae3c454-7598-4df3-9f8e-e8362af020a5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@a5c4023 2022-04-26T06:39:34.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ae3c454-7598-4df3-9f8e-e8362af020a5 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:39:34.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ae3c454-7598-4df3-9f8e-e8362af020a5 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:39:35.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:39:35.374Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:39:35.692Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3ae3c454-7598-4df3-9f8e-e8362af020a5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:39:35.692Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3ae3c454-7598-4df3-9f8e-e8362af020a5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:39:37.541Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70faf7ae-1dbd-4cc1-876e-722cb84ead9c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:39:37.541Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70faf7ae-1dbd-4cc1-876e-722cb84ead9c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4ded0f61 2022-04-26T06:39:37.541Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70faf7ae-1dbd-4cc1-876e-722cb84ead9c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:39:37.542Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70faf7ae-1dbd-4cc1-876e-722cb84ead9c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:39:38.569Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=70faf7ae-1dbd-4cc1-876e-722cb84ead9c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:39:38.569Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=70faf7ae-1dbd-4cc1-876e-722cb84ead9c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:39:39.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=131d9d7e-c955-4788-b652-1b5a683ce905 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:39:39.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=131d9d7e-c955-4788-b652-1b5a683ce905 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2d530e9e 2022-04-26T06:39:39.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=131d9d7e-c955-4788-b652-1b5a683ce905 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:39:39.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=131d9d7e-c955-4788-b652-1b5a683ce905 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:39:40.809Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=131d9d7e-c955-4788-b652-1b5a683ce905 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:39:40.810Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=131d9d7e-c955-4788-b652-1b5a683ce905 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:39:42.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ece42db0-a37a-4ad3-9b70-55b6dde92511 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:39:42.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ece42db0-a37a-4ad3-9b70-55b6dde92511 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5de06ec5 2022-04-26T06:39:42.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ece42db0-a37a-4ad3-9b70-55b6dde92511 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:39:42.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ece42db0-a37a-4ad3-9b70-55b6dde92511 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:39:43.789Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ece42db0-a37a-4ad3-9b70-55b6dde92511 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:39:43.790Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ece42db0-a37a-4ad3-9b70-55b6dde92511 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:39:45.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f6c4255-9b52-44bd-bf75-07f23e3d62f3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:39:45.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f6c4255-9b52-44bd-bf75-07f23e3d62f3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1dfb0e96 2022-04-26T06:39:45.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f6c4255-9b52-44bd-bf75-07f23e3d62f3 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:39:45.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f6c4255-9b52-44bd-bf75-07f23e3d62f3 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:39:46.793Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=9f6c4255-9b52-44bd-bf75-07f23e3d62f3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:39:46.794Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=9f6c4255-9b52-44bd-bf75-07f23e3d62f3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:39:48.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ae15a36-ac9a-4f94-89a0-589246b90420 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:39:48.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ae15a36-ac9a-4f94-89a0-589246b90420 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@24822c72 2022-04-26T06:39:48.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ae15a36-ac9a-4f94-89a0-589246b90420 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:39:48.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ae15a36-ac9a-4f94-89a0-589246b90420 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:39:49.805Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=4ae15a36-ac9a-4f94-89a0-589246b90420 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:39:49.806Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=4ae15a36-ac9a-4f94-89a0-589246b90420 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:39:50.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:39:50.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:39:52.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3aa977a-e315-47df-a479-6e45ca5e9970 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:39:52.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3aa977a-e315-47df-a479-6e45ca5e9970 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7506568 2022-04-26T06:39:52.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3aa977a-e315-47df-a479-6e45ca5e9970 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:39:52.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3aa977a-e315-47df-a479-6e45ca5e9970 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:39:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:39:52.686Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:39:52.686Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:39:53.644Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=b3aa977a-e315-47df-a479-6e45ca5e9970 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:39:53.645Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=b3aa977a-e315-47df-a479-6e45ca5e9970 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:39:55.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c1de594-4bad-4b3f-9ccd-4ef47451b08a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:39:55.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c1de594-4bad-4b3f-9ccd-4ef47451b08a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f7ed992 2022-04-26T06:39:55.634Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c1de594-4bad-4b3f-9ccd-4ef47451b08a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:39:55.634Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c1de594-4bad-4b3f-9ccd-4ef47451b08a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:39:56.657Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=1c1de594-4bad-4b3f-9ccd-4ef47451b08a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:39:56.657Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=1c1de594-4bad-4b3f-9ccd-4ef47451b08a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:39:58.424Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14f6e2dd-9a7e-4326-bb88-2fd0bf04a294 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:39:58.424Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14f6e2dd-9a7e-4326-bb88-2fd0bf04a294 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@bebab85 2022-04-26T06:39:58.425Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14f6e2dd-9a7e-4326-bb88-2fd0bf04a294 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:39:58.425Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14f6e2dd-9a7e-4326-bb88-2fd0bf04a294 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:39:59.024Z [qtp2085002312-20] ERROR o.j.g.database.StandardJanusGraph RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Could not commit transaction [374] due to storage exception in commit org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT1M40S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 103 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 104 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:39:59.025Z [qtp2085002312-20] ERROR o.j.g.database.StandardJanusGraph RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Could not commit transaction [374] due to exception org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT1M40S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 103 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 104 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:39:59.465Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=14f6e2dd-9a7e-4326-bb88-2fd0bf04a294 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:39:59.467Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=14f6e2dd-9a7e-4326-bb88-2fd0bf04a294 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:40:00.134Z [qtp2085002312-20] WARN o.o.sdc.be.servlets.ConsumerServlet RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb #createConsumer('{"consumerName": "clamp", "consumerSalt": "9cd4c3ad2a6f6ce3f3414e68b5157e63", "consumerPassword": "35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495"}', 'Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/consumers)@6d0411d6', 'jh0003'): org.apache.cxf.jaxrs.impl.ResponseImpl@48fd9e6c in 1.20min (too slow!) 2022-04-26T06:40:00.135Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=71727 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:38:48.407Z InvocationID=483d533e-5c2d-4946-a710-685864572e9f LogTimestamp=2022-04-26T06:40:00.134657Z ResponseDesc=Created Exiting. 2022-04-26T06:40:00.135Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:40:00.141Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=42268 PartnerName=kube-probe/1.22 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=ERROR ServerIPAddress=10.233.65.41 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2022-04-26T06:39:17.873Z InvocationID=ccfeca1f-dc2c-4d00-82f7-1842385ef361 ErrorCode=500 LogTimestamp=2022-04-26T06:40:00.141021Z ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:40:00.141Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservfc HTTP/1.1 SC="500" 2022-04-26T06:40:00.151Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=92d96807-bf56-49e1-ad7a-4262686f7c17 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:40:00.151338Z InvocationID=151535f4-e065-4da5-8b82-6ca9d9924fe6 LogTimestamp=2022-04-26T06:40:00.151414Z Entering 2022-04-26T06:40:00.151Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=92d96807-bf56-49e1-ad7a-4262686f7c17 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:40:00.151338Z InvocationID=151535f4-e065-4da5-8b82-6ca9d9924fe6 LogTimestamp=2022-04-26T06:40:00.151414Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:40:00.154Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id vfc 2022-04-26T06:40:01.042Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:40:01.138Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:40:01.338Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:40:01.542Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64ac2352-e529-4c7f-9878-7492919f27df PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:40:01.542Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64ac2352-e529-4c7f-9878-7492919f27df PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1c6c3445 2022-04-26T06:40:01.543Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64ac2352-e529-4c7f-9878-7492919f27df PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:40:01.543Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64ac2352-e529-4c7f-9878-7492919f27df PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:40:01.623Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:40:02.042Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:40:02.569Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=64ac2352-e529-4c7f-9878-7492919f27df ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:40:02.569Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=64ac2352-e529-4c7f-9878-7492919f27df ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:40:02.852Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:40:03.733Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:40:04.538Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7dec4326-99b8-44b0-bd72-00c939271356 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:40:04.538Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7dec4326-99b8-44b0-bd72-00c939271356 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@621004ce 2022-04-26T06:40:04.538Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7dec4326-99b8-44b0-bd72-00c939271356 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:40:04.538Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7dec4326-99b8-44b0-bd72-00c939271356 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:40:04.647Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:40:05.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:40:05.378Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:40:05.534Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:40:05.581Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=7dec4326-99b8-44b0-bd72-00c939271356 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:40:05.581Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=7dec4326-99b8-44b0-bd72-00c939271356 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:40:06.347Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:40:07.239Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f15f801-0577-4955-9fbc-d43a926527d1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:40:07.239Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f15f801-0577-4955-9fbc-d43a926527d1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@545670c0 2022-04-26T06:40:07.239Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f15f801-0577-4955-9fbc-d43a926527d1 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:40:07.240Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f15f801-0577-4955-9fbc-d43a926527d1 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:40:07.704Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:40:08.034Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:40:08.265Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=0f15f801-0577-4955-9fbc-d43a926527d1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:40:08.266Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=0f15f801-0577-4955-9fbc-d43a926527d1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:40:09.640Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:40:09.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81e99ad9-5d88-4a82-88a1-58e3ad44f750 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:40:09.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81e99ad9-5d88-4a82-88a1-58e3ad44f750 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@bfc6085 2022-04-26T06:40:09.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81e99ad9-5d88-4a82-88a1-58e3ad44f750 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:40:09.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81e99ad9-5d88-4a82-88a1-58e3ad44f750 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:40:10.793Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=81e99ad9-5d88-4a82-88a1-58e3ad44f750 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:40:10.793Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=81e99ad9-5d88-4a82-88a1-58e3ad44f750 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:40:12.786Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ed3c4ad-5fbb-44e3-87e6-582cedd81cb7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:40:12.786Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ed3c4ad-5fbb-44e3-87e6-582cedd81cb7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@68179784 2022-04-26T06:40:12.786Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ed3c4ad-5fbb-44e3-87e6-582cedd81cb7 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:40:12.786Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ed3c4ad-5fbb-44e3-87e6-582cedd81cb7 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:40:12.849Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:40:13.801Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=0ed3c4ad-5fbb-44e3-87e6-582cedd81cb7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:40:13.802Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=0ed3c4ad-5fbb-44e3-87e6-582cedd81cb7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:40:15.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01def720-81af-46d8-a9c7-973700d204b3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:40:15.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01def720-81af-46d8-a9c7-973700d204b3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@28ff8da3 2022-04-26T06:40:15.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01def720-81af-46d8-a9c7-973700d204b3 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:40:15.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01def720-81af-46d8-a9c7-973700d204b3 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:40:16.055Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:40:16.809Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=01def720-81af-46d8-a9c7-973700d204b3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:40:16.810Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=01def720-81af-46d8-a9c7-973700d204b3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:40:18.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77535fd0-8602-4c7d-b3aa-8075abe5039c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:40:18.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77535fd0-8602-4c7d-b3aa-8075abe5039c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@72183035 2022-04-26T06:40:18.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77535fd0-8602-4c7d-b3aa-8075abe5039c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:40:18.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77535fd0-8602-4c7d-b3aa-8075abe5039c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:40:19.785Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=77535fd0-8602-4c7d-b3aa-8075abe5039c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:40:19.786Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=77535fd0-8602-4c7d-b3aa-8075abe5039c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:40:20.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:40:20.377Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:40:22.426Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec9d253f-3c91-4b59-b2ac-48c3addb9038 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:40:22.426Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec9d253f-3c91-4b59-b2ac-48c3addb9038 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1fa3d972 2022-04-26T06:40:22.426Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec9d253f-3c91-4b59-b2ac-48c3addb9038 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:40:22.426Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec9d253f-3c91-4b59-b2ac-48c3addb9038 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:40:22.523Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:40:23.465Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ec9d253f-3c91-4b59-b2ac-48c3addb9038 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:40:23.465Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ec9d253f-3c91-4b59-b2ac-48c3addb9038 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:40:25.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3112acc-25c0-4e6f-a505-47edb2687cd3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:40:25.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3112acc-25c0-4e6f-a505-47edb2687cd3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@33b7029a 2022-04-26T06:40:25.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3112acc-25c0-4e6f-a505-47edb2687cd3 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:40:25.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3112acc-25c0-4e6f-a505-47edb2687cd3 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:40:26.665Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e3112acc-25c0-4e6f-a505-47edb2687cd3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:40:26.666Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e3112acc-25c0-4e6f-a505-47edb2687cd3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:40:28.421Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4278bd1-27c8-4162-8f3e-0bab7ee37016 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:40:28.421Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4278bd1-27c8-4162-8f3e-0bab7ee37016 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@57676429 2022-04-26T06:40:28.421Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4278bd1-27c8-4162-8f3e-0bab7ee37016 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:40:28.422Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4278bd1-27c8-4162-8f3e-0bab7ee37016 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:40:28.931Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:40:29.450Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=d4278bd1-27c8-4162-8f3e-0bab7ee37016 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:40:29.450Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=d4278bd1-27c8-4162-8f3e-0bab7ee37016 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:40:31.842Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b067cb4b-d0c7-472a-8779-b3bd6f235437 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:40:31.842Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b067cb4b-d0c7-472a-8779-b3bd6f235437 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@34a3af6b 2022-04-26T06:40:31.842Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b067cb4b-d0c7-472a-8779-b3bd6f235437 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:40:31.843Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b067cb4b-d0c7-472a-8779-b3bd6f235437 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:40:32.883Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=b067cb4b-d0c7-472a-8779-b3bd6f235437 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:40:32.884Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=b067cb4b-d0c7-472a-8779-b3bd6f235437 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:40:34.345Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ccbf594-02c2-49aa-8f37-4600459f933e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:40:34.345Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ccbf594-02c2-49aa-8f37-4600459f933e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7c5de670 2022-04-26T06:40:34.345Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ccbf594-02c2-49aa-8f37-4600459f933e PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:40:34.345Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ccbf594-02c2-49aa-8f37-4600459f933e PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:40:35.342Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:40:35.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:40:35.369Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=2ccbf594-02c2-49aa-8f37-4600459f933e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:40:35.369Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=2ccbf594-02c2-49aa-8f37-4600459f933e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:40:35.385Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:40:37.343Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b15c490-c62a-4644-8a95-dae2309dd122 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:40:37.343Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b15c490-c62a-4644-8a95-dae2309dd122 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@348849c3 2022-04-26T06:40:37.344Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b15c490-c62a-4644-8a95-dae2309dd122 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:40:37.344Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b15c490-c62a-4644-8a95-dae2309dd122 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:40:38.381Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=6b15c490-c62a-4644-8a95-dae2309dd122 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:40:38.381Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=6b15c490-c62a-4644-8a95-dae2309dd122 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:40:39.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76ae688f-92b7-4350-a2d7-3f7648a4da28 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:40:39.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76ae688f-92b7-4350-a2d7-3f7648a4da28 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@bdd066d 2022-04-26T06:40:39.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76ae688f-92b7-4350-a2d7-3f7648a4da28 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:40:39.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76ae688f-92b7-4350-a2d7-3f7648a4da28 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:40:40.813Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=76ae688f-92b7-4350-a2d7-3f7648a4da28 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:40:40.814Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=76ae688f-92b7-4350-a2d7-3f7648a4da28 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:40:41.751Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:40:42.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa5a5e03-dd7d-4387-8e49-b851cf2d1c00 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:40:42.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa5a5e03-dd7d-4387-8e49-b851cf2d1c00 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2b6db8aa 2022-04-26T06:40:42.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa5a5e03-dd7d-4387-8e49-b851cf2d1c00 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:40:42.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa5a5e03-dd7d-4387-8e49-b851cf2d1c00 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:40:43.816Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=aa5a5e03-dd7d-4387-8e49-b851cf2d1c00 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:40:43.817Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=aa5a5e03-dd7d-4387-8e49-b851cf2d1c00 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:40:45.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b476cf91-a51c-4ca5-ba35-148f36b5109d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:40:45.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b476cf91-a51c-4ca5-ba35-148f36b5109d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@50330539 2022-04-26T06:40:45.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b476cf91-a51c-4ca5-ba35-148f36b5109d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:40:45.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b476cf91-a51c-4ca5-ba35-148f36b5109d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:40:46.793Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=b476cf91-a51c-4ca5-ba35-148f36b5109d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:40:46.793Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=b476cf91-a51c-4ca5-ba35-148f36b5109d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:40:48.157Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:40:48.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89289f26-8e05-43da-bd72-df12c5a030da PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:40:48.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89289f26-8e05-43da-bd72-df12c5a030da PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2290964d 2022-04-26T06:40:48.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89289f26-8e05-43da-bd72-df12c5a030da PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:40:48.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89289f26-8e05-43da-bd72-df12c5a030da PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:40:49.800Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=89289f26-8e05-43da-bd72-df12c5a030da ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:40:49.801Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=89289f26-8e05-43da-bd72-df12c5a030da ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:40:50.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:40:50.374Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:40:52.522Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bacedf23-f09e-4f0f-b2f3-6bc5f89a3851 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:40:52.522Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bacedf23-f09e-4f0f-b2f3-6bc5f89a3851 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2da19663 2022-04-26T06:40:52.522Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bacedf23-f09e-4f0f-b2f3-6bc5f89a3851 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:40:52.522Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bacedf23-f09e-4f0f-b2f3-6bc5f89a3851 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:40:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:40:52.686Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:40:52.687Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:40:53.545Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=bacedf23-f09e-4f0f-b2f3-6bc5f89a3851 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:40:53.545Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=bacedf23-f09e-4f0f-b2f3-6bc5f89a3851 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:40:54.622Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:40:55.521Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d78c6ac-85c9-42f8-8ede-3e7ff382ae74 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:40:55.521Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d78c6ac-85c9-42f8-8ede-3e7ff382ae74 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@199da8dd 2022-04-26T06:40:55.522Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d78c6ac-85c9-42f8-8ede-3e7ff382ae74 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:40:55.522Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d78c6ac-85c9-42f8-8ede-3e7ff382ae74 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:40:56.553Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=9d78c6ac-85c9-42f8-8ede-3e7ff382ae74 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:40:56.553Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=9d78c6ac-85c9-42f8-8ede-3e7ff382ae74 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:40:58.231Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2b60c86-88bc-4ad4-a6d0-e2f27afe7996 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:40:58.232Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2b60c86-88bc-4ad4-a6d0-e2f27afe7996 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@474b3460 2022-04-26T06:40:58.232Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2b60c86-88bc-4ad4-a6d0-e2f27afe7996 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:40:58.232Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2b60c86-88bc-4ad4-a6d0-e2f27afe7996 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:40:59.245Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e2b60c86-88bc-4ad4-a6d0-e2f27afe7996 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:40:59.248Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e2b60c86-88bc-4ad4-a6d0-e2f27afe7996 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:41:01.041Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:41:01.532Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d36ab18-1e70-4b19-b591-fb5e94ba1032 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:41:01.533Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d36ab18-1e70-4b19-b591-fb5e94ba1032 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@47952f65 2022-04-26T06:41:01.533Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d36ab18-1e70-4b19-b591-fb5e94ba1032 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:41:01.533Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d36ab18-1e70-4b19-b591-fb5e94ba1032 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:41:02.572Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=7d36ab18-1e70-4b19-b591-fb5e94ba1032 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:41:02.573Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=7d36ab18-1e70-4b19-b591-fb5e94ba1032 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:41:04.329Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad02e3c5-f11e-4711-bb9f-36ad41216ea9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:41:04.330Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad02e3c5-f11e-4711-bb9f-36ad41216ea9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7a9a09b8 2022-04-26T06:41:04.330Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad02e3c5-f11e-4711-bb9f-36ad41216ea9 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:41:04.330Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad02e3c5-f11e-4711-bb9f-36ad41216ea9 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:41:05.353Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ad02e3c5-f11e-4711-bb9f-36ad41216ea9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:41:05.354Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ad02e3c5-f11e-4711-bb9f-36ad41216ea9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:41:05.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:41:05.378Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:41:07.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9c412bd-8be9-45c2-8ba3-f0e2d548d9a0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:41:07.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9c412bd-8be9-45c2-8ba3-f0e2d548d9a0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@72529856 2022-04-26T06:41:07.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9c412bd-8be9-45c2-8ba3-f0e2d548d9a0 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:41:07.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9c412bd-8be9-45c2-8ba3-f0e2d548d9a0 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:41:07.542Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:41:07.681Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:41:08.457Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e9c412bd-8be9-45c2-8ba3-f0e2d548d9a0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:41:08.457Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e9c412bd-8be9-45c2-8ba3-f0e2d548d9a0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:41:09.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86104272-df6f-4c5e-a349-6d88800c6cd3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:41:09.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86104272-df6f-4c5e-a349-6d88800c6cd3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5a0a70d3 2022-04-26T06:41:09.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86104272-df6f-4c5e-a349-6d88800c6cd3 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:41:09.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86104272-df6f-4c5e-a349-6d88800c6cd3 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:41:10.797Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=86104272-df6f-4c5e-a349-6d88800c6cd3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:41:10.798Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=86104272-df6f-4c5e-a349-6d88800c6cd3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:41:12.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed19e61c-5a94-4fac-a20d-d120a98079cb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:41:12.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed19e61c-5a94-4fac-a20d-d120a98079cb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@33341c68 2022-04-26T06:41:12.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed19e61c-5a94-4fac-a20d-d120a98079cb PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:41:12.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed19e61c-5a94-4fac-a20d-d120a98079cb PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:41:13.802Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ed19e61c-5a94-4fac-a20d-d120a98079cb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:41:13.802Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ed19e61c-5a94-4fac-a20d-d120a98079cb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:41:13.949Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:41:15.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75344379-d613-4df5-886d-899a33772a38 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:41:15.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75344379-d613-4df5-886d-899a33772a38 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@71089318 2022-04-26T06:41:15.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75344379-d613-4df5-886d-899a33772a38 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:41:15.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75344379-d613-4df5-886d-899a33772a38 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:41:16.812Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=75344379-d613-4df5-886d-899a33772a38 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:41:16.813Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=75344379-d613-4df5-886d-899a33772a38 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:41:18.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c8fb294-cae8-494f-b643-db8a4b077c26 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:41:18.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c8fb294-cae8-494f-b643-db8a4b077c26 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@223bf9d2 2022-04-26T06:41:18.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c8fb294-cae8-494f-b643-db8a4b077c26 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:41:18.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c8fb294-cae8-494f-b643-db8a4b077c26 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:41:19.821Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=2c8fb294-cae8-494f-b643-db8a4b077c26 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:41:19.821Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=2c8fb294-cae8-494f-b643-db8a4b077c26 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:41:20.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:41:20.387Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:41:22.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e4886e9-e3c9-4189-8c11-26caaa7c4587 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:41:22.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e4886e9-e3c9-4189-8c11-26caaa7c4587 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@c51ca8 2022-04-26T06:41:22.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e4886e9-e3c9-4189-8c11-26caaa7c4587 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:41:22.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e4886e9-e3c9-4189-8c11-26caaa7c4587 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:41:23.657Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=6e4886e9-e3c9-4189-8c11-26caaa7c4587 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:41:23.657Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=6e4886e9-e3c9-4189-8c11-26caaa7c4587 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:41:25.422Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d2c19c6-461f-423d-b78d-3cbb4195160e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:41:25.422Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d2c19c6-461f-423d-b78d-3cbb4195160e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@375e7ca4 2022-04-26T06:41:25.422Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d2c19c6-461f-423d-b78d-3cbb4195160e PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:41:25.422Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d2c19c6-461f-423d-b78d-3cbb4195160e PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:41:26.441Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=9d2c19c6-461f-423d-b78d-3cbb4195160e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:41:26.442Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=9d2c19c6-461f-423d-b78d-3cbb4195160e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:41:26.822Z [qtp2085002312-12] ERROR o.j.g.database.StandardJanusGraph RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Could not commit transaction [377] due to storage exception in commit org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT1M40S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 103 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 104 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:41:26.822Z [qtp2085002312-12] ERROR o.j.g.database.StandardJanusGraph RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Could not commit transaction [377] due to exception org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT1M40S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 103 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 104 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:41:27.630Z [qtp2085002312-12] WARN o.o.sdc.be.servlets.ConsumerServlet RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb #createConsumer('{"consumerName": "vfc", "consumerSalt": "9cd4c3ad2a6f6ce3f3414e68b5157e63", "consumerPassword": "35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495"}', 'Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/consumers)@6d0411d6', 'jh0003'): org.apache.cxf.jaxrs.impl.ResponseImpl@7117023 in 1.46min (too slow!) 2022-04-26T06:41:27.631Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=87479 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:40:00.152Z InvocationID=17818a51-13ab-4977-bbed-01c415b310da LogTimestamp=2022-04-26T06:41:27.631240Z ResponseDesc=Created Exiting. 2022-04-26T06:41:27.631Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:41:27.640Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=189767 PartnerName=kube-probe/1.22 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=ERROR ServerIPAddress=10.233.65.41 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2022-04-26T06:38:17.873Z InvocationID=ee4421f8-1fc7-4f93-b071-217891cfe39a ErrorCode=500 LogTimestamp=2022-04-26T06:41:27.640053Z ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:41:27.640Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userworkflow HTTP/1.1 SC="500" 2022-04-26T06:41:27.647Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=d7383de5-ef7f-468e-aeed-8273a6259a3e InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2022-04-26T06:41:27.647045Z InvocationID=e3495439-6bb2-4fac-945b-eb33cb2a4bb4 LogTimestamp=2022-04-26T06:41:27.647098Z Entering 2022-04-26T06:41:27.647Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=d7383de5-ef7f-468e-aeed-8273a6259a3e InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2022-04-26T06:41:27.647045Z InvocationID=e3495439-6bb2-4fac-945b-eb33cb2a4bb4 LogTimestamp=2022-04-26T06:41:27.647098Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:41:27.650Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:41:27.647Z InvocationID=f57e9c82-e8de-43c0-bcae-f671faf22d20 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id workflow 2022-04-26T06:41:28.322Z [qtp2085002312-14] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:41:27.647Z InvocationID=f57e9c82-e8de-43c0-bcae-f671faf22d20 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:41:28.327Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e27a2f30-4b1f-496b-9a7c-fd246bc4ecfb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:41:28.328Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e27a2f30-4b1f-496b-9a7c-fd246bc4ecfb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2c4f8d5d 2022-04-26T06:41:28.328Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e27a2f30-4b1f-496b-9a7c-fd246bc4ecfb PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:41:28.328Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e27a2f30-4b1f-496b-9a7c-fd246bc4ecfb PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:41:28.423Z [qtp2085002312-14] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:41:27.647Z InvocationID=f57e9c82-e8de-43c0-bcae-f671faf22d20 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:41:28.530Z [qtp2085002312-14] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:41:27.647Z InvocationID=f57e9c82-e8de-43c0-bcae-f671faf22d20 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:41:28.739Z [qtp2085002312-14] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:41:27.647Z InvocationID=f57e9c82-e8de-43c0-bcae-f671faf22d20 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:41:29.221Z [qtp2085002312-14] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:41:27.647Z InvocationID=f57e9c82-e8de-43c0-bcae-f671faf22d20 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:41:29.353Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e27a2f30-4b1f-496b-9a7c-fd246bc4ecfb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:41:29.353Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e27a2f30-4b1f-496b-9a7c-fd246bc4ecfb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:41:30.031Z [qtp2085002312-14] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:41:27.647Z InvocationID=f57e9c82-e8de-43c0-bcae-f671faf22d20 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:41:30.844Z [qtp2085002312-14] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:41:27.647Z InvocationID=f57e9c82-e8de-43c0-bcae-f671faf22d20 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:41:31.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a72b6384-cca4-48c8-9be6-342343d26f75 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:41:31.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a72b6384-cca4-48c8-9be6-342343d26f75 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@15d1a89d 2022-04-26T06:41:31.443Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a72b6384-cca4-48c8-9be6-342343d26f75 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:41:31.443Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a72b6384-cca4-48c8-9be6-342343d26f75 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:41:31.732Z [qtp2085002312-14] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:41:27.647Z InvocationID=f57e9c82-e8de-43c0-bcae-f671faf22d20 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:41:32.457Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=a72b6384-cca4-48c8-9be6-342343d26f75 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:41:32.458Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=a72b6384-cca4-48c8-9be6-342343d26f75 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:41:33.342Z [qtp2085002312-14] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:41:27.647Z InvocationID=f57e9c82-e8de-43c0-bcae-f671faf22d20 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:41:34.328Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cdc7ff9d-5ff1-4376-a7dd-f143830c0f47 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:41:34.329Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cdc7ff9d-5ff1-4376-a7dd-f143830c0f47 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e51d803 2022-04-26T06:41:34.329Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cdc7ff9d-5ff1-4376-a7dd-f143830c0f47 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:41:34.329Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cdc7ff9d-5ff1-4376-a7dd-f143830c0f47 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:41:35.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:41:35.369Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=cdc7ff9d-5ff1-4376-a7dd-f143830c0f47 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:41:35.370Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=cdc7ff9d-5ff1-4376-a7dd-f143830c0f47 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:41:35.380Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:41:36.634Z [qtp2085002312-14] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:41:27.647Z InvocationID=f57e9c82-e8de-43c0-bcae-f671faf22d20 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:41:37.446Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=956444fb-9c21-495e-9d95-ce97c8265096 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:41:37.446Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=956444fb-9c21-495e-9d95-ce97c8265096 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@457b48bf 2022-04-26T06:41:37.447Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=956444fb-9c21-495e-9d95-ce97c8265096 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:41:37.447Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=956444fb-9c21-495e-9d95-ce97c8265096 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:41:38.477Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=956444fb-9c21-495e-9d95-ce97c8265096 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:41:38.478Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=956444fb-9c21-495e-9d95-ce97c8265096 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:41:39.833Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a8b2c895-f2fe-4e2a-9823-ae275cc15e0e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:41:39.834Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a8b2c895-f2fe-4e2a-9823-ae275cc15e0e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3fdbb6be 2022-04-26T06:41:39.834Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a8b2c895-f2fe-4e2a-9823-ae275cc15e0e PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:41:39.834Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a8b2c895-f2fe-4e2a-9823-ae275cc15e0e PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:41:39.840Z [qtp2085002312-14] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:41:27.647Z InvocationID=f57e9c82-e8de-43c0-bcae-f671faf22d20 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:41:40.873Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=a8b2c895-f2fe-4e2a-9823-ae275cc15e0e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:41:40.873Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=a8b2c895-f2fe-4e2a-9823-ae275cc15e0e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:41:42.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d35d23af-8f58-468c-a641-4c61c71b91b0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:41:42.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d35d23af-8f58-468c-a641-4c61c71b91b0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@49d313ae 2022-04-26T06:41:42.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d35d23af-8f58-468c-a641-4c61c71b91b0 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:41:42.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d35d23af-8f58-468c-a641-4c61c71b91b0 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:41:43.816Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=d35d23af-8f58-468c-a641-4c61c71b91b0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:41:43.817Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=d35d23af-8f58-468c-a641-4c61c71b91b0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:41:45.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79a5a544-8a74-4a96-890a-b0013291a83f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:41:45.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79a5a544-8a74-4a96-890a-b0013291a83f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3309d878 2022-04-26T06:41:45.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79a5a544-8a74-4a96-890a-b0013291a83f PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:41:45.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79a5a544-8a74-4a96-890a-b0013291a83f PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:41:46.248Z [qtp2085002312-14] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:41:27.647Z InvocationID=f57e9c82-e8de-43c0-bcae-f671faf22d20 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:41:46.792Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=79a5a544-8a74-4a96-890a-b0013291a83f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:41:46.793Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=79a5a544-8a74-4a96-890a-b0013291a83f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:41:48.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=51d88c31-b263-4ee5-b916-f8b56f00061f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:41:48.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=51d88c31-b263-4ee5-b916-f8b56f00061f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@209d4be4 2022-04-26T06:41:48.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=51d88c31-b263-4ee5-b916-f8b56f00061f PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:41:48.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=51d88c31-b263-4ee5-b916-f8b56f00061f PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:41:49.801Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=51d88c31-b263-4ee5-b916-f8b56f00061f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:41:49.801Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=51d88c31-b263-4ee5-b916-f8b56f00061f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:41:50.368Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:41:50.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:41:52.425Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22382b53-ce14-4b1b-a173-db6d8da725c1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:41:52.426Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22382b53-ce14-4b1b-a173-db6d8da725c1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@23043c9b 2022-04-26T06:41:52.426Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22382b53-ce14-4b1b-a173-db6d8da725c1 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:41:52.426Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22382b53-ce14-4b1b-a173-db6d8da725c1 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:41:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:41:52.686Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:41:52.686Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:41:53.451Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=22382b53-ce14-4b1b-a173-db6d8da725c1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:41:53.451Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=22382b53-ce14-4b1b-a173-db6d8da725c1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:41:55.422Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2af18275-b2e9-4d94-ad24-aac573f97bed PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:41:55.423Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2af18275-b2e9-4d94-ad24-aac573f97bed PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@bacfbf9 2022-04-26T06:41:55.423Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2af18275-b2e9-4d94-ad24-aac573f97bed PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:41:55.423Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2af18275-b2e9-4d94-ad24-aac573f97bed PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:41:56.465Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=2af18275-b2e9-4d94-ad24-aac573f97bed ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:41:56.466Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=2af18275-b2e9-4d94-ad24-aac573f97bed ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:41:58.335Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96f3438c-dda9-400d-bb53-dec15e923280 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:41:58.335Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96f3438c-dda9-400d-bb53-dec15e923280 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4895701d 2022-04-26T06:41:58.335Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96f3438c-dda9-400d-bb53-dec15e923280 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:41:58.335Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96f3438c-dda9-400d-bb53-dec15e923280 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:41:59.369Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=96f3438c-dda9-400d-bb53-dec15e923280 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) 2022-04-26T06:41:59.369Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=96f3438c-dda9-400d-bb53-dec15e923280 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.233.28.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2022-04-26T06:42:01.135Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=33488 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:41:27.647Z InvocationID=f57e9c82-e8de-43c0-bcae-f671faf22d20 LogTimestamp=2022-04-26T06:42:01.135190Z ResponseDesc=Created Exiting. 2022-04-26T06:42:01.136Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:42:01.147Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=43276 PartnerName=kube-probe/1.22 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=ERROR ServerIPAddress=10.233.65.41 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2022-04-26T06:41:17.871Z InvocationID=7e10c036-e455-478e-9fb1-17f31b10a1f3 ErrorCode=500 LogTimestamp=2022-04-26T06:42:01.147664Z ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:42:01.148Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpolicy HTTP/1.1 SC="500" 2022-04-26T06:42:01.152Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=83ad791d-d55d-4da2-992c-2a82efe9e4f1 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:01.152501Z InvocationID=fde31e12-3dc6-4b44-a745-c2c49481b389 LogTimestamp=2022-04-26T06:42:01.152533Z Entering 2022-04-26T06:42:01.152Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=83ad791d-d55d-4da2-992c-2a82efe9e4f1 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:01.152501Z InvocationID=fde31e12-3dc6-4b44-a745-c2c49481b389 LogTimestamp=2022-04-26T06:42:01.152533Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:42:01.155Z [qtp2085002312-13] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:42:01.153Z InvocationID=7d414330-9de8-4117-a67a-b709768de0fe PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id policy 2022-04-26T06:42:01.727Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21b8a70d-8e89-4c01-982f-ea2cec714ab5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:42:01.728Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21b8a70d-8e89-4c01-982f-ea2cec714ab5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@479f80ad 2022-04-26T06:42:01.728Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21b8a70d-8e89-4c01-982f-ea2cec714ab5 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:42:01.728Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21b8a70d-8e89-4c01-982f-ea2cec714ab5 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:42:02.935Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1782 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:42:01.153Z InvocationID=7d414330-9de8-4117-a67a-b709768de0fe LogTimestamp=2022-04-26T06:42:02.935140Z ResponseDesc=Created Exiting. 2022-04-26T06:42:02.935Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:42:02.941Z [qtp2085002312-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:42:02.940884Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:42:02.941Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:42:02.940884Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:42:02.941Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpomba HTTP/1.1 SC="500" 2022-04-26T06:42:02.944Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=57f79747-148d-4ec1-8860-06d32f2453a2 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:02.944769Z InvocationID=e342132e-15a8-4f54-81bf-bb6d09114e9e LogTimestamp=2022-04-26T06:42:02.944800Z Entering 2022-04-26T06:42:02.945Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=57f79747-148d-4ec1-8860-06d32f2453a2 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:02.944769Z InvocationID=e342132e-15a8-4f54-81bf-bb6d09114e9e LogTimestamp=2022-04-26T06:42:02.944800Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:42:02.947Z [qtp2085002312-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:42:02.945Z InvocationID=1ee670f4-be55-4cef-b8a2-339b23ec7aab PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id pomba 2022-04-26T06:42:05.019Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=21b8a70d-8e89-4c01-982f-ea2cec714ab5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:42:05.020Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=21b8a70d-8e89-4c01-982f-ea2cec714ab5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:42:05.368Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:42:05.377Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:42:05.634Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77b8e600-95c7-42ee-9908-74c320cf9a27 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:42:05.634Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77b8e600-95c7-42ee-9908-74c320cf9a27 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@550690b4 2022-04-26T06:42:05.634Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77b8e600-95c7-42ee-9908-74c320cf9a27 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:42:05.637Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77b8e600-95c7-42ee-9908-74c320cf9a27 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:42:05.832Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2886 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:42:02.945Z InvocationID=1ee670f4-be55-4cef-b8a2-339b23ec7aab LogTimestamp=2022-04-26T06:42:05.831655Z ResponseDesc=Created Exiting. 2022-04-26T06:42:05.833Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:42:05.850Z [qtp2085002312-16] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:42:05.850690Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:42:05.851Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:42:05.850690Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:42:05.852Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermulticloud HTTP/1.1 SC="500" 2022-04-26T06:42:05.859Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ffeb964d-1520-4d54-89ef-098eb1fc6687 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:05.858855Z InvocationID=9fde5b3d-42f9-47ca-842e-9d0cad6bd4df LogTimestamp=2022-04-26T06:42:05.858935Z Entering 2022-04-26T06:42:05.859Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ffeb964d-1520-4d54-89ef-098eb1fc6687 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:05.858855Z InvocationID=9fde5b3d-42f9-47ca-842e-9d0cad6bd4df LogTimestamp=2022-04-26T06:42:05.858935Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:42:05.863Z [qtp2085002312-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:42:05.860Z InvocationID=ca031189-a696-490b-91aa-a028c01a2106 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id multicloud 2022-04-26T06:42:07.683Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:42:08.652Z [qtp2085002312-18] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:42:05.860Z InvocationID=ca031189-a696-490b-91aa-a028c01a2106 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:42:08.691Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=77b8e600-95c7-42ee-9908-74c320cf9a27 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:42:08.692Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=77b8e600-95c7-42ee-9908-74c320cf9a27 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:42:08.934Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04355c88-5ade-4e3b-87f3-9688d817e3ab PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:42:08.934Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04355c88-5ade-4e3b-87f3-9688d817e3ab PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@43a2322b 2022-04-26T06:42:08.934Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04355c88-5ade-4e3b-87f3-9688d817e3ab PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:42:08.934Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04355c88-5ade-4e3b-87f3-9688d817e3ab PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:42:10.730Z [qtp2085002312-18] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:42:05.860Z InvocationID=ca031189-a696-490b-91aa-a028c01a2106 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:42:11.992Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=04355c88-5ade-4e3b-87f3-9688d817e3ab ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:42:11.992Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=04355c88-5ade-4e3b-87f3-9688d817e3ab ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:42:12.151Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a95a1994-50b9-460f-8681-46e26fe7052a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:42:12.151Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a95a1994-50b9-460f-8681-46e26fe7052a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@14d95b60 2022-04-26T06:42:12.151Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a95a1994-50b9-460f-8681-46e26fe7052a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:42:12.151Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a95a1994-50b9-460f-8681-46e26fe7052a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:42:12.762Z [qtp2085002312-18] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:42:05.860Z InvocationID=ca031189-a696-490b-91aa-a028c01a2106 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:42:14.818Z [qtp2085002312-18] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:42:05.860Z InvocationID=ca031189-a696-490b-91aa-a028c01a2106 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:42:15.202Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=a95a1994-50b9-460f-8681-46e26fe7052a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:42:15.202Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=a95a1994-50b9-460f-8681-46e26fe7052a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:42:15.350Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ca3be50-be52-43ba-9804-1d5f8811adba PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:42:15.350Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ca3be50-be52-43ba-9804-1d5f8811adba PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2422c9a3 2022-04-26T06:42:15.351Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ca3be50-be52-43ba-9804-1d5f8811adba PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:42:15.351Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ca3be50-be52-43ba-9804-1d5f8811adba PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:42:16.879Z [qtp2085002312-18] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:42:05.860Z InvocationID=ca031189-a696-490b-91aa-a028c01a2106 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:42:18.399Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3ca3be50-be52-43ba-9804-1d5f8811adba ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:42:18.400Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3ca3be50-be52-43ba-9804-1d5f8811adba ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:42:18.546Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40db7225-8dc8-40f1-846f-34248afe6791 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:42:18.546Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40db7225-8dc8-40f1-846f-34248afe6791 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@505d72f1 2022-04-26T06:42:18.546Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40db7225-8dc8-40f1-846f-34248afe6791 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:42:18.547Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40db7225-8dc8-40f1-846f-34248afe6791 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:42:18.935Z [qtp2085002312-18] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:42:05.860Z InvocationID=ca031189-a696-490b-91aa-a028c01a2106 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:42:20.368Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:42:20.378Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:42:21.120Z [qtp2085002312-18] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:42:05.860Z InvocationID=ca031189-a696-490b-91aa-a028c01a2106 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:42:21.591Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=40db7225-8dc8-40f1-846f-34248afe6791 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:42:21.592Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=40db7225-8dc8-40f1-846f-34248afe6791 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:42:22.520Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eb5d55e8-32eb-415e-a2b9-a9eb40a735f1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:42:22.521Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eb5d55e8-32eb-415e-a2b9-a9eb40a735f1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8d8bd3c 2022-04-26T06:42:22.521Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eb5d55e8-32eb-415e-a2b9-a9eb40a735f1 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:42:22.521Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eb5d55e8-32eb-415e-a2b9-a9eb40a735f1 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:42:23.332Z [qtp2085002312-18] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:42:05.860Z InvocationID=ca031189-a696-490b-91aa-a028c01a2106 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:42:25.541Z [qtp2085002312-18] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:42:05.860Z InvocationID=ca031189-a696-490b-91aa-a028c01a2106 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:42:25.563Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=eb5d55e8-32eb-415e-a2b9-a9eb40a735f1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:42:25.564Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=eb5d55e8-32eb-415e-a2b9-a9eb40a735f1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:42:26.325Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f5285b7-cf91-49c4-b139-f62033340506 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:42:26.325Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f5285b7-cf91-49c4-b139-f62033340506 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@347f26bd 2022-04-26T06:42:26.325Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f5285b7-cf91-49c4-b139-f62033340506 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:42:26.325Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f5285b7-cf91-49c4-b139-f62033340506 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:42:28.021Z [qtp2085002312-18] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:42:05.860Z InvocationID=ca031189-a696-490b-91aa-a028c01a2106 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:42:29.366Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=4f5285b7-cf91-49c4-b139-f62033340506 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:42:29.367Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=4f5285b7-cf91-49c4-b139-f62033340506 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:42:30.320Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10b32441-4ae2-48d8-9673-96703ad92cb3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:42:30.320Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10b32441-4ae2-48d8-9673-96703ad92cb3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6bb49970 2022-04-26T06:42:30.321Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10b32441-4ae2-48d8-9673-96703ad92cb3 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:42:30.321Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10b32441-4ae2-48d8-9673-96703ad92cb3 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:42:30.426Z [qtp2085002312-18] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:42:05.860Z InvocationID=ca031189-a696-490b-91aa-a028c01a2106 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:42:33.145Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=27284 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:42:05.860Z InvocationID=ca031189-a696-490b-91aa-a028c01a2106 LogTimestamp=2022-04-26T06:42:33.144347Z ResponseDesc=Created Exiting. 2022-04-26T06:42:33.145Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:42:33.155Z [qtp2085002312-18] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:42:33.155823Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2022-04-26T06:42:33.156Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2022-04-26T06:42:33.155823Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:42:33.156Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercds HTTP/1.1 SC="500" 2022-04-26T06:42:33.161Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=4527a0e4-d352-4113-97a5-7478cbb0dd0a InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:33.160960Z InvocationID=8b9e33fe-1130-4f7e-b52b-06209747433b LogTimestamp=2022-04-26T06:42:33.161001Z Entering 2022-04-26T06:42:33.161Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=4527a0e4-d352-4113-97a5-7478cbb0dd0a InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:33.160960Z InvocationID=8b9e33fe-1130-4f7e-b52b-06209747433b LogTimestamp=2022-04-26T06:42:33.161001Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:42:33.164Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:42:33.161Z InvocationID=6e360052-cb33-4c9b-bae1-53a2921ce43d PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id cds 2022-04-26T06:42:33.360Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=10b32441-4ae2-48d8-9673-96703ad92cb3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:42:33.360Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=10b32441-4ae2-48d8-9673-96703ad92cb3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:42:34.242Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60e588e9-957d-4b8f-a347-703056d4cf21 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:42:34.242Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60e588e9-957d-4b8f-a347-703056d4cf21 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@59780982 2022-04-26T06:42:34.242Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60e588e9-957d-4b8f-a347-703056d4cf21 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:42:34.242Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60e588e9-957d-4b8f-a347-703056d4cf21 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:42:35.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:42:35.400Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:42:36.653Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=3492 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:42:33.161Z InvocationID=6e360052-cb33-4c9b-bae1-53a2921ce43d LogTimestamp=2022-04-26T06:42:36.653247Z ResponseDesc=Created Exiting. 2022-04-26T06:42:36.654Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:42:36.659Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=18785 PartnerName=kube-probe/1.22 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=ERROR ServerIPAddress=10.233.65.41 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2022-04-26T06:42:17.874Z InvocationID=6a1e6a9d-a49f-457f-bd2b-4117b639467f ErrorCode=500 LogTimestamp=2022-04-26T06:42:36.659340Z ResponseDesc=Internal Server Error Exiting. 2022-04-26T06:42:36.659Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermodeling HTTP/1.1 SC="500" 2022-04-26T06:42:36.663Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=8003ddc0-3341-4e8c-90f9-563191d46483 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:36.663348Z InvocationID=d708a102-752c-49e7-88cd-e9301a8c5128 LogTimestamp=2022-04-26T06:42:36.663387Z Entering 2022-04-26T06:42:36.663Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=8003ddc0-3341-4e8c-90f9-563191d46483 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServiceName=v1/consumers ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:36.663348Z InvocationID=d708a102-752c-49e7-88cd-e9301a8c5128 LogTimestamp=2022-04-26T06:42:36.663387Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/consumers HTTP/1.1 2022-04-26T06:42:36.665Z [qtp2085002312-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:42:36.663Z InvocationID=02bc7274-1507-4f1d-bb99-a363e59a40d4 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lock resource with id modeling 2022-04-26T06:42:37.291Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=60e588e9-957d-4b8f-a347-703056d4cf21 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:42:37.291Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=60e588e9-957d-4b8f-a347-703056d4cf21 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:42:38.027Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d44b2fd0-5330-43e3-8acb-60068f66ac65 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:42:38.028Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d44b2fd0-5330-43e3-8acb-60068f66ac65 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@776f14a6 2022-04-26T06:42:38.028Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d44b2fd0-5330-43e3-8acb-60068f66ac65 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:42:38.028Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d44b2fd0-5330-43e3-8acb-60068f66ac65 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:42:39.348Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:42:36.663Z InvocationID=02bc7274-1507-4f1d-bb99-a363e59a40d4 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:42:41.059Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=d44b2fd0-5330-43e3-8acb-60068f66ac65 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:42:41.059Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=d44b2fd0-5330-43e3-8acb-60068f66ac65 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:42:41.179Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03ff3cc0-5fc0-422a-8446-48dd87129c27 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:42:41.179Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03ff3cc0-5fc0-422a-8446-48dd87129c27 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6aedfb1 2022-04-26T06:42:41.179Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03ff3cc0-5fc0-422a-8446-48dd87129c27 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:42:41.179Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03ff3cc0-5fc0-422a-8446-48dd87129c27 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:42:41.404Z [qtp2085002312-17] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:42:36.663Z InvocationID=02bc7274-1507-4f1d-bb99-a363e59a40d4 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.handleTransaction(ConsumerOperation.java:152) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:141) at org.openecomp.sdc.be.model.operations.impl.ConsumerOperation.updateCredentials(ConsumerOperation.java:122) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.updateConsumer(ConsumerBusinessLogic.java:281) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:104) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:90) at jdk.internal.reflect.GeneratedMethodAccessor492.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:42:42.846Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=6183 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/consumers EntryTimestamp=2022-04-26T06:42:36.663Z InvocationID=02bc7274-1507-4f1d-bb99-a363e59a40d4 LogTimestamp=2022-04-26T06:42:42.846259Z ResponseDesc=Created Exiting. 2022-04-26T06:42:42.847Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2022-04-26T06:42:43.034Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=63f827c6-5d1a-47f1-8085-71862ba6913a InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServiceName=v1/user/jh0003 ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:43.034631Z InvocationID=7f4bcf0f-74ea-4bc9-b741-6b9c2f1cfb6b LogTimestamp=2022-04-26T06:42:43.034697Z Entering 2022-04-26T06:42:43.035Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=63f827c6-5d1a-47f1-8085-71862ba6913a InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServiceName=v1/user/jh0003 ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:43.034631Z InvocationID=7f4bcf0f-74ea-4bc9-b741-6b9c2f1cfb6b LogTimestamp=2022-04-26T06:42:43.034697Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2022-04-26T06:42:43.037Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=2 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/user/jh0003 EntryTimestamp=2022-04-26T06:42:43.035Z InvocationID=d3cefd6c-eab0-4d7f-b1af-c64b8ccc0ba1 LogTimestamp=2022-04-26T06:42:43.037755Z ResponseDesc=OK Exiting. 2022-04-26T06:42:43.038Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2022-04-26T06:42:43.107Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=03ff3cc0-5fc0-422a-8446-48dd87129c27 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:42:43.107Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=03ff3cc0-5fc0-422a-8446-48dd87129c27 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:42:43.252Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=42a230a9-7165-421e-acf4-9e8bf01cdd44 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:42:43.253Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=42a230a9-7165-421e-acf4-9e8bf01cdd44 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4ad90dd4 2022-04-26T06:42:43.253Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=42a230a9-7165-421e-acf4-9e8bf01cdd44 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:42:43.253Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=42a230a9-7165-421e-acf4-9e8bf01cdd44 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:42:43.382Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=e47c3cfc-2e75-4cac-b9fa-e9d04a5710f1 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/screen ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:43.382438Z InvocationID=8e38ec5a-1480-40e9-9a16-01521cdf03a0 LogTimestamp=2022-04-26T06:42:43.382499Z Entering 2022-04-26T06:42:43.383Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=e47c3cfc-2e75-4cac-b9fa-e9d04a5710f1 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/screen ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:43.382438Z InvocationID=8e38ec5a-1480-40e9-9a16-01521cdf03a0 LogTimestamp=2022-04-26T06:42:43.382499Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 GET /sdc2/rest/v1/screen HTTP/1.1 2022-04-26T06:42:43.832Z [qtp2085002312-20] INFO o.o.s.b.m.j.o.ToscaElementOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/screen EntryTimestamp=2022-04-26T06:42:43.383Z InvocationID=cc8e0b6f-f628-47a1-8f14-6b02ab1fd8ea PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb time to fetch all catalog elements: StopWatch '': running time = 446952857 ns\n---------------------------------------------\nns % Task name\n---------------------------------------------\n446952857 100% \n 2022-04-26T06:42:43.864Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=480 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/screen EntryTimestamp=2022-04-26T06:42:43.383Z InvocationID=cc8e0b6f-f628-47a1-8f14-6b02ab1fd8ea LogTimestamp=2022-04-26T06:42:43.863747Z ResponseDesc=OK Exiting. 2022-04-26T06:42:43.864Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/screen HTTP/1.1 SC="200" 2022-04-26T06:42:43.877Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=e08def70-bca1-4a90-b16d-07be0d21581c InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServiceName=v1/catalog/model ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:43.877391Z InvocationID=de0e4a0a-d961-44ff-86f9-df2be45be72e LogTimestamp=2022-04-26T06:42:43.877454Z Entering 2022-04-26T06:42:43.878Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=e08def70-bca1-4a90-b16d-07be0d21581c InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServiceName=v1/catalog/model ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:43.877391Z InvocationID=de0e4a0a-d961-44ff-86f9-df2be45be72e LogTimestamp=2022-04-26T06:42:43.877454Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 GET /sdc2/rest/v1/catalog/model HTTP/1.1 2022-04-26T06:42:43.883Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/model EntryTimestamp=2022-04-26T06:42:43.878Z InvocationID=5c4c0670-ac39-4346-a668-28219950c0b9 LogTimestamp=2022-04-26T06:42:43.883236Z ResponseDesc=OK Exiting. 2022-04-26T06:42:43.883Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/catalog/model HTTP/1.1 SC="200" 2022-04-26T06:42:43.898Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=eb7bea94-54b5-4296-aa0d-3400aae67e82 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServiceName=v1/catalog/model ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:43.898407Z InvocationID=71e11c38-b51e-4839-ae50-f4e6c73ec0ae LogTimestamp=2022-04-26T06:42:43.898490Z Entering 2022-04-26T06:42:43.899Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=eb7bea94-54b5-4296-aa0d-3400aae67e82 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServiceName=v1/catalog/model ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:43.898407Z InvocationID=71e11c38-b51e-4839-ae50-f4e6c73ec0ae LogTimestamp=2022-04-26T06:42:43.898490Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/catalog/model HTTP/1.1 2022-04-26T06:42:44.457Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=558 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/model EntryTimestamp=2022-04-26T06:42:43.899Z InvocationID=27ced3e0-7e70-42e1-a6b8-3c4eb861af78 LogTimestamp=2022-04-26T06:42:44.457007Z ResponseDesc=Created Exiting. 2022-04-26T06:42:44.457Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/model HTTP/1.1 SC="201" 2022-04-26T06:42:44.473Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=239a4b88-dc47-4863-a2b1-22672f3808d7 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:44.473215Z InvocationID=d4eada47-5ca6-48df-b8f6-67133555e01d LogTimestamp=2022-04-26T06:42:44.473334Z Entering 2022-04-26T06:42:44.474Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=239a4b88-dc47-4863-a2b1-22672f3808d7 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:44.473215Z InvocationID=d4eada47-5ca6-48df-b8f6-67133555e01d LogTimestamp=2022-04-26T06:42:44.473334Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2022-04-26T06:42:44.542Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.Root.datatype to dao for create 2022-04-26T06:42:44.544Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.548Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : ETSI SOL001 v2.5.1.integer.datatype to dao for create 2022-04-26T06:42:44.550Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.553Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : ETSI SOL001 v2.5.1.string.datatype to dao for create 2022-04-26T06:42:44.556Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.558Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : ETSI SOL001 v2.5.1.boolean.datatype to dao for create 2022-04-26T06:42:44.563Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.565Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : ETSI SOL001 v2.5.1.float.datatype to dao for create 2022-04-26T06:42:44.568Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.571Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : ETSI SOL001 v2.5.1.list.datatype to dao for create 2022-04-26T06:42:44.574Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.577Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : ETSI SOL001 v2.5.1.map.datatype to dao for create 2022-04-26T06:42:44.591Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.595Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : ETSI SOL001 v2.5.1.json.datatype to dao for create 2022-04-26T06:42:44.599Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.604Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : ETSI SOL001 v2.5.1.scalar-unit.datatype to dao for create 2022-04-26T06:42:44.607Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.614Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : ETSI SOL001 v2.5.1.scalar-unit.size.datatype to dao for create 2022-04-26T06:42:44.617Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.621Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : ETSI SOL001 v2.5.1.scalar-unit.time.datatype to dao for create 2022-04-26T06:42:44.623Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.626Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : ETSI SOL001 v2.5.1.scalar-unit.frequency.datatype to dao for create 2022-04-26T06:42:44.628Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.631Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.L2AddressData.datatype to dao for create 2022-04-26T06:42:44.641Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.644Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.L3AddressData.datatype to dao for create 2022-04-26T06:42:44.656Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.659Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.AddressData.datatype to dao for create 2022-04-26T06:42:44.682Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.686Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.LogicalNodeData.datatype to dao for create 2022-04-26T06:42:44.692Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.695Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VirtualNetworkInterfaceRequirements.datatype to dao for create 2022-04-26T06:42:44.717Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.721Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.ConnectivityType.datatype to dao for create 2022-04-26T06:42:44.730Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.733Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.RequestedAdditionalCapability.datatype to dao for create 2022-04-26T06:42:44.750Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.753Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VirtualMemory.datatype to dao for create 2022-04-26T06:42:44.775Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.779Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VirtualCpuPinning.datatype to dao for create 2022-04-26T06:42:44.790Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.795Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VirtualCpu.datatype to dao for create 2022-04-26T06:42:44.820Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.822Z [qtp2085002312-12] WARN o.o.s.b.c.impl.DataTypeImportManager RequestID=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Creating data type tosca.datatypes.nfv.VnfcConfigurableProperties which derived from abstract data type with no properties 2022-04-26T06:42:44.823Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfcConfigurableProperties.datatype to dao for create 2022-04-26T06:42:44.827Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.829Z [qtp2085002312-12] WARN o.o.s.b.c.impl.DataTypeImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Creating data type tosca.datatypes.nfv.VnfcAdditionalConfigurableProperties which derived from abstract data type with no properties 2022-04-26T06:42:44.830Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfcAdditionalConfigurableProperties.datatype to dao for create 2022-04-26T06:42:44.833Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.837Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VduProfile.datatype to dao for create 2022-04-26T06:42:44.851Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.854Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.L2ProtocolData.datatype to dao for create 2022-04-26T06:42:44.869Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.873Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.L3ProtocolData.datatype to dao for create 2022-04-26T06:42:44.898Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.901Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VirtualLinkProtocolData.datatype to dao for create 2022-04-26T06:42:44.921Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.925Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.IpAllocationPool.datatype to dao for create 2022-04-26T06:42:44.931Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.934Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.InstantiationLevel.datatype to dao for create 2022-04-26T06:42:44.940Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.943Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VduLevel.datatype to dao for create 2022-04-26T06:42:44.948Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.950Z [qtp2085002312-12] WARN o.o.s.b.c.impl.DataTypeImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Creating data type tosca.datatypes.nfv.VnfInstantiateOperationConfiguration which derived from abstract data type with no properties 2022-04-26T06:42:44.951Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfInstantiateOperationConfiguration.datatype to dao for create 2022-04-26T06:42:44.952Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.956Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfScaleOperationConfiguration.datatype to dao for create 2022-04-26T06:42:44.964Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.969Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfScaleToLevelOperationConfiguration.datatype to dao for create 2022-04-26T06:42:44.976Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.979Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfHealOperationConfiguration.datatype to dao for create 2022-04-26T06:42:44.985Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:44.989Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfTerminateOperationConfiguration.datatype to dao for create 2022-04-26T06:42:44.998Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.002Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfOperateOperationConfiguration.datatype to dao for create 2022-04-26T06:42:45.009Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.012Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.ScaleInfo.datatype to dao for create 2022-04-26T06:42:45.022Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.025Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.ScalingAspect.datatype to dao for create 2022-04-26T06:42:45.036Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.040Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.LinkBitrateRequirements.datatype to dao for create 2022-04-26T06:42:45.048Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.052Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.Qos.datatype to dao for create 2022-04-26T06:42:45.063Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.066Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfConfigurableProperties.datatype to dao for create 2022-04-26T06:42:45.073Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.075Z [qtp2085002312-12] WARN o.o.s.b.c.impl.DataTypeImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Creating data type tosca.datatypes.nfv.VnfAdditionalConfigurableProperties which derived from abstract data type with no properties 2022-04-26T06:42:45.076Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfAdditionalConfigurableProperties.datatype to dao for create 2022-04-26T06:42:45.083Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.087Z [qtp2085002312-12] WARN o.o.s.b.c.impl.DataTypeImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Creating data type tosca.datatypes.nfv.VnfInfoModifiableAttributes which derived from abstract data type with no properties 2022-04-26T06:42:45.088Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfInfoModifiableAttributes.datatype to dao for create 2022-04-26T06:42:45.095Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.098Z [qtp2085002312-12] WARN o.o.s.b.c.impl.DataTypeImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Creating data type tosca.datatypes.nfv.VnfInfoModifiableAttributesExtensions which derived from abstract data type with no properties 2022-04-26T06:42:45.100Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfInfoModifiableAttributesExtensions.datatype to dao for create 2022-04-26T06:42:45.104Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.106Z [qtp2085002312-12] WARN o.o.s.b.c.impl.DataTypeImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Creating data type tosca.datatypes.nfv.VnfInfoModifiableAttributesMetadata which derived from abstract data type with no properties 2022-04-26T06:42:45.108Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfInfoModifiableAttributesMetadata.datatype to dao for create 2022-04-26T06:42:45.115Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.118Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.CpProtocolData.datatype to dao for create 2022-04-26T06:42:45.130Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.133Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.SwImageData.datatype to dao for create 2022-04-26T06:42:45.163Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.167Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VirtualBlockStorageData.datatype to dao for create 2022-04-26T06:42:45.181Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.184Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VirtualObjectStorageData.datatype to dao for create 2022-04-26T06:42:45.190Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.193Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VirtualFileStorageData.datatype to dao for create 2022-04-26T06:42:45.201Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.204Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VirtualLinkBitrateLevel.datatype to dao for create 2022-04-26T06:42:45.213Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.215Z [qtp2085002312-12] WARN o.o.s.b.c.impl.DataTypeImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Creating data type tosca.datatypes.nfv.VnfOperationAdditionalParameters which derived from abstract data type with no properties 2022-04-26T06:42:45.216Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfOperationAdditionalParameters.datatype to dao for create 2022-04-26T06:42:45.220Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.222Z [qtp2085002312-12] WARN o.o.s.b.c.impl.DataTypeImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Creating data type tosca.datatypes.nfv.VnfChangeFlavourOperationConfiguration which derived from abstract data type with no properties 2022-04-26T06:42:45.223Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfChangeFlavourOperationConfiguration.datatype to dao for create 2022-04-26T06:42:45.227Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.230Z [qtp2085002312-12] WARN o.o.s.b.c.impl.DataTypeImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Creating data type tosca.datatypes.nfv.VnfChangeExtConnectivityOperationConfiguration which derived from abstract data type with no properties 2022-04-26T06:42:45.231Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfChangeExtConnectivityOperationConfiguration.datatype to dao for create 2022-04-26T06:42:45.233Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.236Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfMonitoringParameter.datatype to dao for create 2022-04-26T06:42:45.248Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.265Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfcMonitoringParameter.datatype to dao for create 2022-04-26T06:42:45.276Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.376Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VirtualLinkMonitoringParameter.datatype to dao for create 2022-04-26T06:42:45.398Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.403Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.UriAuthority.datatype to dao for create 2022-04-26T06:42:45.413Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.417Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.UriComponents.datatype to dao for create 2022-04-26T06:42:45.437Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.441Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.InterfaceDetails.datatype to dao for create 2022-04-26T06:42:45.451Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.456Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfProfile.datatype to dao for create 2022-04-26T06:42:45.464Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.469Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.ServiceAvailability.datatype to dao for create 2022-04-26T06:42:45.476Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.487Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.NsVirtualLinkQos.datatype to dao for create 2022-04-26T06:42:45.494Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.501Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VlProfile.datatype to dao for create 2022-04-26T06:42:45.525Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.530Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.NsVlProfile.datatype to dao for create 2022-04-26T06:42:45.561Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.565Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 send DATA_TYPE : ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfLcmOperationsConfiguration.datatype to dao for create 2022-04-26T06:42:45.615Z [qtp2085002312-12] INFO o.o.s.b.m.o.impl.PropertyOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=DataType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:45.618Z [qtp2085002312-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 all DataTypeDefinition(derivedFrom=null, constraints=null, properties=[PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfLcmOperationsConfiguration.datatype.operate, type=tosca.datatypes.nfv.VnfOperateOperationConfiguration, required=false, defaultValue=null, description=Configuration parameters for the OperateVnf operation, schema=null, password=false, name=operate, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, model=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=null, metadata=null), name=operate, constraints=null]], PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfLcmOperationsConfiguration.datatype.scale_to_level, type=tosca.datatypes.nfv.VnfScaleToLevelOperationConfiguration, required=false, defaultValue=null, description=Configuration parameters for the ScaleVnfToLevel operation, schema=null, password=false, name=scale_to_level, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, model=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=null, metadata=null), name=scale_to_level, constraints=null]], PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfLcmOperationsConfiguration.datatype.change_flavour, type=tosca.datatypes.nfv.VnfChangeFlavourOperationConfiguration, required=false, defaultValue=null, description=Configuration parameters for the changeVnfFlavourOpConfig operation, schema=null, password=false, name=change_flavour, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, model=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=null, metadata=null), name=change_flavour, constraints=null]], PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfLcmOperationsConfiguration.datatype.heal, type=tosca.datatypes.nfv.VnfHealOperationConfiguration, required=false, defaultValue=null, description=Configuration parameters for the HealVnf operation, schema=null, password=false, name=heal, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, model=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=null, metadata=null), name=heal, constraints=null]], PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfLcmOperationsConfiguration.datatype.scale, type=tosca.datatypes.nfv.VnfScaleOperationConfiguration, required=false, defaultValue=null, description=Configuration parameters for the ScaleVnf operation, schema=null, password=false, name=scale, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, model=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=null, metadata=null), name=scale, constraints=null]], PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfLcmOperationsConfiguration.datatype.change_ext_connectivity, type=tosca.datatypes.nfv.VnfChangeExtConnectivityOperationConfiguration, required=false, defaultValue=null, description=Configuration parameters for the changeExtVnfConnectivityOpConfig operation, schema=null, password=false, name=change_ext_connectivity, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, model=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=null, metadata=null), name=change_ext_connectivity, constraints=null]], PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfLcmOperationsConfiguration.datatype.terminate, type=tosca.datatypes.nfv.VnfTerminateOperationConfiguration, required=false, defaultValue=null, description=Configuration parameters for the TerminateVnf operation, schema=null, password=false, name=terminate, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, model=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=null, metadata=null), name=terminate, constraints=null]], PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=ETSI SOL001 v2.5.1.tosca.datatypes.nfv.VnfLcmOperationsConfiguration.datatype.instantiate, type=tosca.datatypes.nfv.VnfInstantiateOperationConfiguration, required=false, defaultValue=null, description=Configuration parameters for the InstantiateVnf operation, schema=null, password=false, name=instantiate, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, model=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=null, metadata=null), name=instantiate, constraints=null]]]) were created successfully!!! 2022-04-26T06:42:46.259Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=42a230a9-7165-421e-acf4-9e8bf01cdd44 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:42:46.259Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=42a230a9-7165-421e-acf4-9e8bf01cdd44 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:42:46.260Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=42a230a9-7165-421e-acf4-9e8bf01cdd44 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:42:46.260Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=42a230a9-7165-421e-acf4-9e8bf01cdd44 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:42:46.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fce8fab6-281d-480c-ac5c-bc791a22ffe9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:42:46.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fce8fab6-281d-480c-ac5c-bc791a22ffe9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4a54a8f4 2022-04-26T06:42:46.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fce8fab6-281d-480c-ac5c-bc791a22ffe9 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:42:46.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fce8fab6-281d-480c-ac5c-bc791a22ffe9 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:42:48.443Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=3969 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:42:44.474Z InvocationID=df1a5ecf-690e-478c-bd79-7e54542f0d1e ErrorCode=500 LogTimestamp=2022-04-26T06:42:48.443676Z ResponseDesc=Created Exiting. 2022-04-26T06:42:48.444Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="201" 2022-04-26T06:42:48.455Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=8c8a2742-42d1-4fc2-88c6-3069c1449622 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServiceName=v1/catalog/uploadType/capability ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:48.455446Z InvocationID=09b3fd46-90c2-4dae-b2fc-17d2765b437f LogTimestamp=2022-04-26T06:42:48.455497Z Entering 2022-04-26T06:42:48.456Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=8c8a2742-42d1-4fc2-88c6-3069c1449622 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServiceName=v1/catalog/uploadType/capability ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:48.455446Z InvocationID=09b3fd46-90c2-4dae-b2fc-17d2765b437f LogTimestamp=2022-04-26T06:42:48.455497Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/catalog/uploadType/capability HTTP/1.1 2022-04-26T06:42:48.468Z [qtp2085002312-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:42:48.456Z InvocationID=0abedf5b-7304-4484-8f56-73b5778828fc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : ETSI SOL001 v2.5.1.tosca.capabilities.Root to dao for create 2022-04-26T06:42:48.470Z [qtp2085002312-18] INFO o.o.s.b.m.o.i.CapabilityTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:42:48.456Z InvocationID=0abedf5b-7304-4484-8f56-73b5778828fc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=CapabilityType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:48.473Z [qtp2085002312-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:42:48.456Z InvocationID=0abedf5b-7304-4484-8f56-73b5778828fc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : ETSI SOL001 v2.5.1.tosca.capabilities.Node to dao for create 2022-04-26T06:42:48.475Z [qtp2085002312-18] INFO o.o.s.b.m.o.i.CapabilityTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:42:48.456Z InvocationID=0abedf5b-7304-4484-8f56-73b5778828fc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=CapabilityType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:48.483Z [qtp2085002312-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:42:48.456Z InvocationID=0abedf5b-7304-4484-8f56-73b5778828fc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : ETSI SOL001 v2.5.1.tosca.capabilities.nfv.VirtualBindable to dao for create 2022-04-26T06:42:48.485Z [qtp2085002312-18] INFO o.o.s.b.m.o.i.CapabilityTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:42:48.456Z InvocationID=0abedf5b-7304-4484-8f56-73b5778828fc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=CapabilityType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:48.492Z [qtp2085002312-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:42:48.456Z InvocationID=0abedf5b-7304-4484-8f56-73b5778828fc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : ETSI SOL001 v2.5.1.tosca.capabilities.nfv.VirtualLinkable to dao for create 2022-04-26T06:42:48.494Z [qtp2085002312-18] INFO o.o.s.b.m.o.i.CapabilityTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:42:48.456Z InvocationID=0abedf5b-7304-4484-8f56-73b5778828fc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=CapabilityType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:48.498Z [qtp2085002312-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:42:48.456Z InvocationID=0abedf5b-7304-4484-8f56-73b5778828fc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : ETSI SOL001 v2.5.1.tosca.capabilities.nfv.VirtualCompute to dao for create 2022-04-26T06:42:48.538Z [qtp2085002312-18] INFO o.o.s.b.m.o.i.CapabilityTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:42:48.456Z InvocationID=0abedf5b-7304-4484-8f56-73b5778828fc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=CapabilityType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:48.543Z [qtp2085002312-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:42:48.456Z InvocationID=0abedf5b-7304-4484-8f56-73b5778828fc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : ETSI SOL001 v2.5.1.tosca.capabilities.nfv.VirtualStorage to dao for create 2022-04-26T06:42:48.545Z [qtp2085002312-18] INFO o.o.s.b.m.o.i.CapabilityTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:42:48.456Z InvocationID=0abedf5b-7304-4484-8f56-73b5778828fc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=CapabilityType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:48.549Z [qtp2085002312-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:42:48.456Z InvocationID=0abedf5b-7304-4484-8f56-73b5778828fc PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb all CapabilityTypeDefinition(super=CapabilityTypeDataDefinition [uniqueId=ETSI SOL001 v2.5.1.tosca.capabilities.nfv.VirtualStorage, description=Describes the attachment capabilities related to Vdu.Storage, type=tosca.capabilities.nfv.VirtualStorage, validSourceTypes=null, version=null, creationTime=1650955368545, modificationTime=1650955368545], derivedFrom=tosca.capabilities.Root, properties=null, model=ETSI SOL001 v2.5.1) were created successfully!!! 2022-04-26T06:42:49.164Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=708 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:42:48.456Z InvocationID=0abedf5b-7304-4484-8f56-73b5778828fc LogTimestamp=2022-04-26T06:42:49.164239Z ResponseDesc=Created Exiting. 2022-04-26T06:42:49.165Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/capability HTTP/1.1 SC="201" 2022-04-26T06:42:49.175Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=2e47d38d-9414-40b5-a9dd-8705878ac7e9 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServiceName=v1/catalog/uploadType/relationship ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:49.175499Z InvocationID=62c70095-05f6-440d-b3fa-93f6b7161d12 LogTimestamp=2022-04-26T06:42:49.175555Z Entering 2022-04-26T06:42:49.176Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=2e47d38d-9414-40b5-a9dd-8705878ac7e9 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServiceName=v1/catalog/uploadType/relationship ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:49.175499Z InvocationID=62c70095-05f6-440d-b3fa-93f6b7161d12 LogTimestamp=2022-04-26T06:42:49.175555Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/catalog/uploadType/relationship HTTP/1.1 2022-04-26T06:42:49.185Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:42:49.176Z InvocationID=8052e665-2539-4808-b903-7012aa188877 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : ETSI SOL001 v2.5.1.tosca.relationships.Root to dao for create 2022-04-26T06:42:49.220Z [qtp2085002312-20] INFO o.o.s.b.m.o.i.RelationshipTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:42:49.176Z InvocationID=8052e665-2539-4808-b903-7012aa188877 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=RelationshipType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:49.466Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=fce8fab6-281d-480c-ac5c-bc791a22ffe9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:42:49.467Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=fce8fab6-281d-480c-ac5c-bc791a22ffe9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:42:49.736Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:42:49.176Z InvocationID=8052e665-2539-4808-b903-7012aa188877 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : ETSI SOL001 v2.5.1.tosca.relationships.DependsOn to dao for create 2022-04-26T06:42:49.738Z [qtp2085002312-20] INFO o.o.s.b.m.o.i.RelationshipTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:42:49.176Z InvocationID=8052e665-2539-4808-b903-7012aa188877 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=RelationshipType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:50.119Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0c20b62-516a-4855-be6d-fee0d71960bf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:42:50.119Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0c20b62-516a-4855-be6d-fee0d71960bf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1816a230 2022-04-26T06:42:50.119Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0c20b62-516a-4855-be6d-fee0d71960bf PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:42:50.119Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0c20b62-516a-4855-be6d-fee0d71960bf PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:42:50.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:42:50.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:42:50.535Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:42:49.176Z InvocationID=8052e665-2539-4808-b903-7012aa188877 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : ETSI SOL001 v2.5.1.tosca.relationships.nfv.VirtualBindsTo to dao for create 2022-04-26T06:42:50.538Z [qtp2085002312-20] INFO o.o.s.b.m.o.i.RelationshipTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:42:49.176Z InvocationID=8052e665-2539-4808-b903-7012aa188877 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=RelationshipType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:51.233Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:42:49.176Z InvocationID=8052e665-2539-4808-b903-7012aa188877 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : ETSI SOL001 v2.5.1.tosca.relationships.nfv.VirtualLinksTo to dao for create 2022-04-26T06:42:51.236Z [qtp2085002312-20] INFO o.o.s.b.m.o.i.RelationshipTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:42:49.176Z InvocationID=8052e665-2539-4808-b903-7012aa188877 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=RelationshipType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:51.930Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:42:49.176Z InvocationID=8052e665-2539-4808-b903-7012aa188877 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : ETSI SOL001 v2.5.1.tosca.relationships.nfv.AttachesTo to dao for create 2022-04-26T06:42:52.018Z [qtp2085002312-20] INFO o.o.s.b.m.o.i.RelationshipTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:42:49.176Z InvocationID=8052e665-2539-4808-b903-7012aa188877 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=RelationshipType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:52.627Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:42:49.176Z InvocationID=8052e665-2539-4808-b903-7012aa188877 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb all RelationshipTypeDefinition(super=RelationshipTypeDataDefinition [uniqueId=ETSI SOL001 v2.5.1.tosca.relationships.nfv.AttachesTo, description=Represents an association relationship between the Vdu.Compute and one of the node types, Vdu.VirtualBlockStorage, Vdu.VirtualObjectStorage or Vdu.VirtualFileStorage, type=tosca.relationships.nfv.AttachesTo, validSourceTypes=null, version=null, originUI=null, creationTime=1650955372017, modificationTime=1650955372017], derivedFrom=tosca.relationships.Root, properties=null, model=ETSI SOL001 v2.5.1) were created successfully!!! 2022-04-26T06:42:52.630Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=3454 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:42:49.176Z InvocationID=8052e665-2539-4808-b903-7012aa188877 LogTimestamp=2022-04-26T06:42:52.630390Z ResponseDesc=Created Exiting. 2022-04-26T06:42:52.631Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/relationship HTTP/1.1 SC="201" 2022-04-26T06:42:52.638Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=cc1280ca-2cea-47ac-8862-f42809d917bf InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServiceName=v1/catalog/uploadType/interfaceLifecycle ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:52.638348Z InvocationID=64d254f7-106c-4d11-bdf9-3111d48c7578 LogTimestamp=2022-04-26T06:42:52.638403Z Entering 2022-04-26T06:42:52.638Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=cc1280ca-2cea-47ac-8862-f42809d917bf InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServiceName=v1/catalog/uploadType/interfaceLifecycle ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:42:52.638348Z InvocationID=64d254f7-106c-4d11-bdf9-3111d48c7578 LogTimestamp=2022-04-26T06:42:52.638403Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/catalog/uploadType/interfaceLifecycle HTTP/1.1 2022-04-26T06:42:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:42:52.686Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:42:52.686Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:42:52.722Z [qtp2085002312-17] INFO o.o.s.b.c.i.InterfaceLifecycleTypeImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2022-04-26T06:42:52.639Z InvocationID=c96695fe-ddd5-41df-9c12-86d23ce4cfb1 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send interfaceDefinition tosca.interfaces.Root to dao for create 2022-04-26T06:42:52.728Z [qtp2085002312-17] INFO o.o.s.b.m.o.i.InterfaceLifecycleOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2022-04-26T06:42:52.639Z InvocationID=c96695fe-ddd5-41df-9c12-86d23ce4cfb1 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=Interface, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:52.879Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=b0c20b62-516a-4855-be6d-fee0d71960bf ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:42:52.879Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=b0c20b62-516a-4855-be6d-fee0d71960bf ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:42:53.417Z [qtp2085002312-17] INFO o.o.s.b.c.i.InterfaceLifecycleTypeImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2022-04-26T06:42:52.639Z InvocationID=c96695fe-ddd5-41df-9c12-86d23ce4cfb1 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send interfaceDefinition tosca.interfaces.nfv.Vnflcm to dao for create 2022-04-26T06:42:53.830Z [qtp2085002312-17] INFO o.o.s.b.m.o.i.InterfaceLifecycleOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2022-04-26T06:42:52.639Z InvocationID=c96695fe-ddd5-41df-9c12-86d23ce4cfb1 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=Interface, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:42:53.924Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57927a89-7fc5-448a-bc4f-18b427a6774f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:42:53.924Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57927a89-7fc5-448a-bc4f-18b427a6774f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@755fa3e3 2022-04-26T06:42:53.924Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57927a89-7fc5-448a-bc4f-18b427a6774f PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:42:53.924Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57927a89-7fc5-448a-bc4f-18b427a6774f PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:42:56.928Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=57927a89-7fc5-448a-bc4f-18b427a6774f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:42:56.928Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=57927a89-7fc5-448a-bc4f-18b427a6774f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:42:56.929Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=57927a89-7fc5-448a-bc4f-18b427a6774f ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:42:56.929Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=57927a89-7fc5-448a-bc4f-18b427a6774f ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:42:57.418Z [qtp2085002312-17] INFO o.o.s.b.c.i.InterfaceLifecycleTypeImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2022-04-26T06:42:52.639Z InvocationID=c96695fe-ddd5-41df-9c12-86d23ce4cfb1 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send interfaceDefinition tosca.interfaces.nfv.Nslcm to dao for create 2022-04-26T06:42:57.727Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bddc4e8a-83ef-4d20-90bd-ed92dbab298e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:42:57.727Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bddc4e8a-83ef-4d20-90bd-ed92dbab298e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@72859575 2022-04-26T06:42:57.727Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bddc4e8a-83ef-4d20-90bd-ed92dbab298e PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:42:57.727Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bddc4e8a-83ef-4d20-90bd-ed92dbab298e PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:42:57.738Z [qtp2085002312-17] INFO o.o.s.b.m.o.i.InterfaceLifecycleOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2022-04-26T06:42:52.639Z InvocationID=c96695fe-ddd5-41df-9c12-86d23ce4cfb1 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=Interface, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:00.763Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=bddc4e8a-83ef-4d20-90bd-ed92dbab298e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:43:00.764Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=bddc4e8a-83ef-4d20-90bd-ed92dbab298e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:43:00.929Z [qtp2085002312-17] INFO o.o.s.b.c.i.InterfaceLifecycleTypeImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2022-04-26T06:42:52.639Z InvocationID=c96695fe-ddd5-41df-9c12-86d23ce4cfb1 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb lifecycle types were created successfully!!! 2022-04-26T06:43:00.933Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=8294 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2022-04-26T06:42:52.639Z InvocationID=c96695fe-ddd5-41df-9c12-86d23ce4cfb1 LogTimestamp=2022-04-26T06:43:00.933166Z ResponseDesc=Created Exiting. 2022-04-26T06:43:00.933Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/interfaceLifecycle HTTP/1.1 SC="201" 2022-04-26T06:43:00.941Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f60fc812-4265-4aa0-a42f-52b907494403 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServiceName=v1/catalog/uploadType/artifactTypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:43:00.941252Z InvocationID=ec4eb925-a86c-4a73-b2bf-f70b8129a67a LogTimestamp=2022-04-26T06:43:00.941304Z Entering 2022-04-26T06:43:00.941Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f60fc812-4265-4aa0-a42f-52b907494403 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServiceName=v1/catalog/uploadType/artifactTypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:43:00.941252Z InvocationID=ec4eb925-a86c-4a73-b2bf-f70b8129a67a LogTimestamp=2022-04-26T06:43:00.941304Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/catalog/uploadType/artifactTypes HTTP/1.1 2022-04-26T06:43:01.119Z [qtp2085002312-16] INFO o.o.s.b.m.o.i.ArtifactTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/artifactTypes EntryTimestamp=2022-04-26T06:43:00.944Z InvocationID=7fb80441-15fd-4383-a917-41ee5500a9e9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=ArtifactType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:01.532Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44f1aed1-cc27-4664-b320-ec3900a91584 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:43:01.532Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44f1aed1-cc27-4664-b320-ec3900a91584 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@58d1bf7a 2022-04-26T06:43:01.532Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44f1aed1-cc27-4664-b320-ec3900a91584 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:43:01.532Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44f1aed1-cc27-4664-b320-ec3900a91584 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:43:02.027Z [qtp2085002312-16] INFO o.o.s.b.m.o.i.ArtifactTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/artifactTypes EntryTimestamp=2022-04-26T06:43:00.944Z InvocationID=7fb80441-15fd-4383-a917-41ee5500a9e9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=ArtifactType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:02.931Z [qtp2085002312-16] INFO o.o.s.b.m.o.i.ArtifactTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/artifactTypes EntryTimestamp=2022-04-26T06:43:00.944Z InvocationID=7fb80441-15fd-4383-a917-41ee5500a9e9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=ArtifactType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:04.032Z [qtp2085002312-16] INFO o.o.s.b.m.o.i.ArtifactTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/artifactTypes EntryTimestamp=2022-04-26T06:43:00.944Z InvocationID=7fb80441-15fd-4383-a917-41ee5500a9e9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=ArtifactType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:04.566Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=44f1aed1-cc27-4664-b320-ec3900a91584 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:43:04.567Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=44f1aed1-cc27-4664-b320-ec3900a91584 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:43:04.648Z [qtp2085002312-16] INFO o.o.s.b.m.o.i.ArtifactTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/artifactTypes EntryTimestamp=2022-04-26T06:43:00.944Z InvocationID=7fb80441-15fd-4383-a917-41ee5500a9e9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=ArtifactType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:05.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:43:05.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:43:05.444Z [qtp2085002312-16] INFO o.o.s.b.m.o.i.ArtifactTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/artifactTypes EntryTimestamp=2022-04-26T06:43:00.944Z InvocationID=7fb80441-15fd-4383-a917-41ee5500a9e9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=ArtifactType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:05.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=587af12d-3a13-4039-8039-22619987d8d2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:43:05.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=587af12d-3a13-4039-8039-22619987d8d2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@ee94673 2022-04-26T06:43:05.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=587af12d-3a13-4039-8039-22619987d8d2 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:43:05.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=587af12d-3a13-4039-8039-22619987d8d2 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:43:06.329Z [qtp2085002312-16] INFO o.o.s.b.m.o.i.ArtifactTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/artifactTypes EntryTimestamp=2022-04-26T06:43:00.944Z InvocationID=7fb80441-15fd-4383-a917-41ee5500a9e9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=ArtifactType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:07.042Z [qtp2085002312-16] INFO o.o.s.b.m.o.i.ArtifactTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/artifactTypes EntryTimestamp=2022-04-26T06:43:00.944Z InvocationID=7fb80441-15fd-4383-a917-41ee5500a9e9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=ArtifactType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:07.669Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:43:07.734Z [qtp2085002312-16] INFO o.o.s.b.m.o.i.ArtifactTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/artifactTypes EntryTimestamp=2022-04-26T06:43:00.944Z InvocationID=7fb80441-15fd-4383-a917-41ee5500a9e9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=ArtifactType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:08.564Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=587af12d-3a13-4039-8039-22619987d8d2 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:43:08.565Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=587af12d-3a13-4039-8039-22619987d8d2 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:43:09.240Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2f81168-8952-4364-8d07-2e642180ae14 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:43:09.240Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2f81168-8952-4364-8d07-2e642180ae14 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@796adf08 2022-04-26T06:43:09.240Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2f81168-8952-4364-8d07-2e642180ae14 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:43:09.241Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2f81168-8952-4364-8d07-2e642180ae14 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:43:10.335Z [qtp2085002312-16] WARN o.j.d.l.c.ConsistentKeyLocker RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/artifactTypes EntryTimestamp=2022-04-26T06:43:00.944Z InvocationID=7fb80441-15fd-4383-a917-41ee5500a9e9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary storage failure while checking locks org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:291) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.getSliceWithRetries(ConsistentKeyLocker.java:464) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock(ConsistentKeyLocker.java:426) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock(ConsistentKeyLocker.java:118) at org.janusgraph.diskstorage.locking.AbstractLocker.checkLocks(AbstractLocker.java:351) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.checkAllLocks(ExpectedValueCheckingTransaction.java:175) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.prepareForMutations(ExpectedValueCheckingTransaction.java:154) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:72) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ArtifactTypeOperation.createArtifactType(ArtifactTypeOperation.java:127) at org.openecomp.sdc.be.model.operations.impl.ArtifactTypeOperation.createArtifactType(ArtifactTypeOperation.java:82) at org.openecomp.sdc.be.components.impl.ArtifactTypeImportManager.lambda$createArtifactTypesByDao$1(ArtifactTypeImportManager.java:83) at java.base/java.util.ArrayList.forEach(Unknown Source) at org.openecomp.sdc.be.components.impl.ArtifactTypeImportManager.createArtifactTypesByDao(ArtifactTypeImportManager.java:83) at org.openecomp.sdc.be.components.impl.ArtifactTypeImportManager.createArtifactTypes(ArtifactTypeImportManager.java:62) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadArtifactTypes$4(TypesUploadServlet.java:199) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadArtifactTypes$5(TypesUploadServlet.java:199) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadArtifactTypes_aroundBody6(TypesUploadServlet.java:200) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure7.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadArtifactTypes(TypesUploadServlet.java:198) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadArtifactTypes() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency ALL (3 responses were required but only 2 replica responded) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency ALL (3 responses were required but only 2 replica responded) at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124) at com.datastax.driver.core.Responses$Error.asException(Responses.java:169) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency ALL (3 responses were required but only 2 replica responded) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:90) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:43:10.359Z [qtp2085002312-16] INFO o.o.s.b.m.o.i.ArtifactTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/artifactTypes EntryTimestamp=2022-04-26T06:43:00.944Z InvocationID=7fb80441-15fd-4383-a917-41ee5500a9e9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=ArtifactType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:10.888Z [qtp2085002312-16] INFO o.o.s.b.m.o.i.ArtifactTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/artifactTypes EntryTimestamp=2022-04-26T06:43:00.944Z InvocationID=7fb80441-15fd-4383-a917-41ee5500a9e9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=ArtifactType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:11.201Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=e2f81168-8952-4364-8d07-2e642180ae14 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:43:11.202Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=e2f81168-8952-4364-8d07-2e642180ae14 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:43:11.346Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e948212-336d-428a-8b6a-23793f967bf9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:43:11.346Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e948212-336d-428a-8b6a-23793f967bf9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@278e375d 2022-04-26T06:43:11.346Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e948212-336d-428a-8b6a-23793f967bf9 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:43:11.347Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e948212-336d-428a-8b6a-23793f967bf9 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:43:11.437Z [qtp2085002312-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=10493 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/artifactTypes EntryTimestamp=2022-04-26T06:43:00.944Z InvocationID=7fb80441-15fd-4383-a917-41ee5500a9e9 LogTimestamp=2022-04-26T06:43:11.437164Z ResponseDesc=Created Exiting. 2022-04-26T06:43:11.437Z [qtp2085002312-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/artifactTypes HTTP/1.1 SC="201" 2022-04-26T06:43:11.447Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=9f1484cb-87f6-47e5-bcf2-2536d4c7fc9f InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServiceName=v1/catalog/upload/resource/import ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:43:11.446743Z InvocationID=28359d81-b09c-450e-98df-3c9e09f5bd81 LogTimestamp=2022-04-26T06:43:11.446811Z Entering 2022-04-26T06:43:11.447Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=9f1484cb-87f6-47e5-bcf2-2536d4c7fc9f InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServiceName=v1/catalog/upload/resource/import ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:43:11.446743Z InvocationID=28359d81-b09c-450e-98df-3c9e09f5bd81 LogTimestamp=2022-04-26T06:43:11.446811Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/catalog/upload/resource/import HTTP/1.1 2022-04-26T06:43:11.593Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:11.593Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:11.593Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:11.593Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:11.607Z [qtp2085002312-17] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Create Resource - validateLifecycleTypesCreate ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=400 Interface tosca.interfaces.node.lifecycle.standard required is missing 2022-04-26T06:43:11.607Z [qtp2085002312-17] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Create Resource - validateLifecycleTypesCreate ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Operation towards database failed. 2022-04-26T06:43:11.608Z [qtp2085002312-17] WARN o.o.s.e.AbstractSdncException RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=900 2022-04-26T06:43:14.159Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9e60d3e-f10c-42a5-b1ff-fc410f73aade PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:43:14.159Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9e60d3e-f10c-42a5-b1ff-fc410f73aade PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@27672dd4 2022-04-26T06:43:14.159Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9e60d3e-f10c-42a5-b1ff-fc410f73aade PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:43:14.159Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9e60d3e-f10c-42a5-b1ff-fc410f73aade PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:43:15.637Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Root" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e8bfa945-3aeb-43c9-83af-20b4e6e814a6" INVARIANT_UUID = "817320af-2ae4-45f2-bebd-71de2a6e0f36" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "e7778221-fad7-4a67-b6f8-0be852376d70.assettoscatemplate" CURR_ARTIFACT_UUID = "e7778221-fad7-4a67-b6f8-0be852376d70.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Root-template.yml,0,e7778221-fad7-4a67-b6f8-0be852376d70.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=726dfbe6-2da3-47ef-a928-4c7141c91e35 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=900 LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:15.637Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Root" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e8bfa945-3aeb-43c9-83af-20b4e6e814a6" INVARIANT_UUID = "817320af-2ae4-45f2-bebd-71de2a6e0f36" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "e7778221-fad7-4a67-b6f8-0be852376d70.assettoscatemplate" CURR_ARTIFACT_UUID = "e7778221-fad7-4a67-b6f8-0be852376d70.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Root-template.yml,0,e7778221-fad7-4a67-b6f8-0be852376d70.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=726dfbe6-2da3-47ef-a928-4c7141c91e35 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=900 Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:15.638Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Root" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e8bfa945-3aeb-43c9-83af-20b4e6e814a6" INVARIANT_UUID = "817320af-2ae4-45f2-bebd-71de2a6e0f36" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "e7778221-fad7-4a67-b6f8-0be852376d70.assettoscatemplate" CURR_ARTIFACT_UUID = "e7778221-fad7-4a67-b6f8-0be852376d70.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Root-template.yml,0,e7778221-fad7-4a67-b6f8-0be852376d70.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=726dfbe6-2da3-47ef-a928-4c7141c91e35 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=900 Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:15.638Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Root" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e8bfa945-3aeb-43c9-83af-20b4e6e814a6" INVARIANT_UUID = "817320af-2ae4-45f2-bebd-71de2a6e0f36" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "e7778221-fad7-4a67-b6f8-0be852376d70.assettoscatemplate" CURR_ARTIFACT_UUID = "e7778221-fad7-4a67-b6f8-0be852376d70.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Root-template.yml,0,e7778221-fad7-4a67-b6f8-0be852376d70.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=726dfbe6-2da3-47ef-a928-4c7141c91e35 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=900 Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:15.648Z [qtp2085002312-17] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Root" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e8bfa945-3aeb-43c9-83af-20b4e6e814a6" INVARIANT_UUID = "817320af-2ae4-45f2-bebd-71de2a6e0f36" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "e7778221-fad7-4a67-b6f8-0be852376d70.assettoscatemplate" CURR_ARTIFACT_UUID = "e7778221-fad7-4a67-b6f8-0be852376d70.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Root-template.yml,0,e7778221-fad7-4a67-b6f8-0be852376d70.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Create Resource - validateLifecycleTypesCreate ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=726dfbe6-2da3-47ef-a928-4c7141c91e35 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=400 Interface tosca.interfaces.nfv.vnflcm required is missing 2022-04-26T06:43:15.648Z [qtp2085002312-17] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Root" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e8bfa945-3aeb-43c9-83af-20b4e6e814a6" INVARIANT_UUID = "817320af-2ae4-45f2-bebd-71de2a6e0f36" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "e7778221-fad7-4a67-b6f8-0be852376d70.assettoscatemplate" CURR_ARTIFACT_UUID = "e7778221-fad7-4a67-b6f8-0be852376d70.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Root-template.yml,0,e7778221-fad7-4a67-b6f8-0be852376d70.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Create Resource - validateLifecycleTypesCreate ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=726dfbe6-2da3-47ef-a928-4c7141c91e35 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Operation towards database failed. 2022-04-26T06:43:15.648Z [qtp2085002312-17] WARN o.o.s.e.AbstractSdncException AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Root" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e8bfa945-3aeb-43c9-83af-20b4e6e814a6" INVARIANT_UUID = "817320af-2ae4-45f2-bebd-71de2a6e0f36" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "e7778221-fad7-4a67-b6f8-0be852376d70.assettoscatemplate" CURR_ARTIFACT_UUID = "e7778221-fad7-4a67-b6f8-0be852376d70.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Root-template.yml,0,e7778221-fad7-4a67-b6f8-0be852376d70.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=726dfbe6-2da3-47ef-a928-4c7141c91e35 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=900 2022-04-26T06:43:15.696Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "97f12e98-4c4c-4866-9844-2546108e6cd2" INVARIANT_UUID = "0c18d256-6ae1-423a-9c2a-fc972e45e4c4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VNF" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=97f12e98-4c4c-4866-9844-2546108e6cd2 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=900 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:43:15.697Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "97f12e98-4c4c-4866-9844-2546108e6cd2" INVARIANT_UUID = "0c18d256-6ae1-423a-9c2a-fc972e45e4c4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VNF" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=97f12e98-4c4c-4866-9844-2546108e6cd2 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:43:15.726Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "97f12e98-4c4c-4866-9844-2546108e6cd2" INVARIANT_UUID = "0c18d256-6ae1-423a-9c2a-fc972e45e4c4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "7a591bb3-f5ee-4270-a5ca-1b9eb3c3c01d.assettoscatemplate" CURR_ARTIFACT_UUID = "7a591bb3-f5ee-4270-a5ca-1b9eb3c3c01d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnf-template.yml,0,7a591bb3-f5ee-4270-a5ca-1b9eb3c3c01d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=7316e44b-3c4b-45d0-82c7-dccca5216917 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Starting to create Csar for component VNF 2022-04-26T06:43:15.750Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "97f12e98-4c4c-4866-9844-2546108e6cd2" INVARIANT_UUID = "0c18d256-6ae1-423a-9c2a-fc972e45e4c4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "7a591bb3-f5ee-4270-a5ca-1b9eb3c3c01d.assettoscatemplate" CURR_ARTIFACT_UUID = "7a591bb3-f5ee-4270-a5ca-1b9eb3c3c01d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnf-template.yml,0,7a591bb3-f5ee-4270-a5ca-1b9eb3c3c01d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=7316e44b-3c4b-45d0-82c7-dccca5216917 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Ended create Csar for component VNF 2022-04-26T06:43:15.795Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "97f12e98-4c4c-4866-9844-2546108e6cd2" INVARIANT_UUID = "0c18d256-6ae1-423a-9c2a-fc972e45e4c4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "7a591bb3-f5ee-4270-a5ca-1b9eb3c3c01d.assettoscacsar" CURR_ARTIFACT_UUID = "7a591bb3-f5ee-4270-a5ca-1b9eb3c3c01d.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Vnf-csar.csar,0,7a591bb3-f5ee-4270-a5ca-1b9eb3c3c01d.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=e36446b4-4374-45bd-b5ba-c3730a3a5d80 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:15.795Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "97f12e98-4c4c-4866-9844-2546108e6cd2" INVARIANT_UUID = "0c18d256-6ae1-423a-9c2a-fc972e45e4c4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "7a591bb3-f5ee-4270-a5ca-1b9eb3c3c01d.assettoscacsar" CURR_ARTIFACT_UUID = "7a591bb3-f5ee-4270-a5ca-1b9eb3c3c01d.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Vnf-csar.csar,0,7a591bb3-f5ee-4270-a5ca-1b9eb3c3c01d.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=e36446b4-4374-45bd-b5ba-c3730a3a5d80 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:15.795Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "97f12e98-4c4c-4866-9844-2546108e6cd2" INVARIANT_UUID = "0c18d256-6ae1-423a-9c2a-fc972e45e4c4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "7a591bb3-f5ee-4270-a5ca-1b9eb3c3c01d.assettoscacsar" CURR_ARTIFACT_UUID = "7a591bb3-f5ee-4270-a5ca-1b9eb3c3c01d.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Vnf-csar.csar,0,7a591bb3-f5ee-4270-a5ca-1b9eb3c3c01d.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=e36446b4-4374-45bd-b5ba-c3730a3a5d80 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:15.795Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "97f12e98-4c4c-4866-9844-2546108e6cd2" INVARIANT_UUID = "0c18d256-6ae1-423a-9c2a-fc972e45e4c4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "7a591bb3-f5ee-4270-a5ca-1b9eb3c3c01d.assettoscacsar" CURR_ARTIFACT_UUID = "7a591bb3-f5ee-4270-a5ca-1b9eb3c3c01d.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Vnf-csar.csar,0,7a591bb3-f5ee-4270-a5ca-1b9eb3c3c01d.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=e36446b4-4374-45bd-b5ba-c3730a3a5d80 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:15.828Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Cp" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "77d0db70-3aec-47f4-84f1-5aac7a552ea3" INVARIANT_UUID = "694a00f7-59d5-4945-9089-a8b649b34336" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Cp" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=77d0db70-3aec-47f4-84f1-5aac7a552ea3 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:43:15.828Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Cp" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "77d0db70-3aec-47f4-84f1-5aac7a552ea3" INVARIANT_UUID = "694a00f7-59d5-4945-9089-a8b649b34336" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Cp" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=77d0db70-3aec-47f4-84f1-5aac7a552ea3 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:43:15.849Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Cp" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "77d0db70-3aec-47f4-84f1-5aac7a552ea3" INVARIANT_UUID = "694a00f7-59d5-4945-9089-a8b649b34336" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "f5bef95d-8ad5-47d9-867a-9f043361e225.assettoscatemplate" CURR_ARTIFACT_UUID = "f5bef95d-8ad5-47d9-867a-9f043361e225.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Cp-template.yml,0,f5bef95d-8ad5-47d9-867a-9f043361e225.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=5816b8c2-1ca6-4c0c-abcd-e58413ecaa22 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Starting to create Csar for component Cp 2022-04-26T06:43:15.870Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Cp" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "77d0db70-3aec-47f4-84f1-5aac7a552ea3" INVARIANT_UUID = "694a00f7-59d5-4945-9089-a8b649b34336" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "f5bef95d-8ad5-47d9-867a-9f043361e225.assettoscatemplate" CURR_ARTIFACT_UUID = "f5bef95d-8ad5-47d9-867a-9f043361e225.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Cp-template.yml,0,f5bef95d-8ad5-47d9-867a-9f043361e225.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=5816b8c2-1ca6-4c0c-abcd-e58413ecaa22 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Ended create Csar for component Cp 2022-04-26T06:43:15.915Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Cp" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "77d0db70-3aec-47f4-84f1-5aac7a552ea3" INVARIANT_UUID = "694a00f7-59d5-4945-9089-a8b649b34336" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "f5bef95d-8ad5-47d9-867a-9f043361e225.assettoscacsar" CURR_ARTIFACT_UUID = "f5bef95d-8ad5-47d9-867a-9f043361e225.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Cp-csar.csar,0,f5bef95d-8ad5-47d9-867a-9f043361e225.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=dcf1c612-913f-413c-973e-f43f0db635fc TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:15.916Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Cp" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "77d0db70-3aec-47f4-84f1-5aac7a552ea3" INVARIANT_UUID = "694a00f7-59d5-4945-9089-a8b649b34336" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "f5bef95d-8ad5-47d9-867a-9f043361e225.assettoscacsar" CURR_ARTIFACT_UUID = "f5bef95d-8ad5-47d9-867a-9f043361e225.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Cp-csar.csar,0,f5bef95d-8ad5-47d9-867a-9f043361e225.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=dcf1c612-913f-413c-973e-f43f0db635fc TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:15.916Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Cp" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "77d0db70-3aec-47f4-84f1-5aac7a552ea3" INVARIANT_UUID = "694a00f7-59d5-4945-9089-a8b649b34336" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "f5bef95d-8ad5-47d9-867a-9f043361e225.assettoscacsar" CURR_ARTIFACT_UUID = "f5bef95d-8ad5-47d9-867a-9f043361e225.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Cp-csar.csar,0,f5bef95d-8ad5-47d9-867a-9f043361e225.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=dcf1c612-913f-413c-973e-f43f0db635fc TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:15.917Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Cp" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "77d0db70-3aec-47f4-84f1-5aac7a552ea3" INVARIANT_UUID = "694a00f7-59d5-4945-9089-a8b649b34336" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "f5bef95d-8ad5-47d9-867a-9f043361e225.assettoscacsar" CURR_ARTIFACT_UUID = "f5bef95d-8ad5-47d9-867a-9f043361e225.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Cp-csar.csar,0,f5bef95d-8ad5-47d9-867a-9f043361e225.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=dcf1c612-913f-413c-973e-f43f0db635fc TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:15.952Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "a47f71f3-17e4-41c0-97c5-acfc7e58d48b" INVARIANT_UUID = "cd9d9d77-294e-4c33-bc4b-9d028bdb4b0d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VnfExtCp" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=a47f71f3-17e4-41c0-97c5-acfc7e58d48b TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:43:15.952Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "a47f71f3-17e4-41c0-97c5-acfc7e58d48b" INVARIANT_UUID = "cd9d9d77-294e-4c33-bc4b-9d028bdb4b0d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VnfExtCp" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=a47f71f3-17e4-41c0-97c5-acfc7e58d48b TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:43:15.979Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "a47f71f3-17e4-41c0-97c5-acfc7e58d48b" INVARIANT_UUID = "cd9d9d77-294e-4c33-bc4b-9d028bdb4b0d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5da04caf-9597-4760-aa71-b788eff1c03f.assettoscatemplate" CURR_ARTIFACT_UUID = "5da04caf-9597-4760-aa71-b788eff1c03f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VnfExternalCp-template.yml,0,5da04caf-9597-4760-aa71-b788eff1c03f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=b1fced86-bac1-420f-b9b5-428bbf4d01eb TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Starting to create Csar for component VNF External CP 2022-04-26T06:43:15.999Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "a47f71f3-17e4-41c0-97c5-acfc7e58d48b" INVARIANT_UUID = "cd9d9d77-294e-4c33-bc4b-9d028bdb4b0d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5da04caf-9597-4760-aa71-b788eff1c03f.assettoscatemplate" CURR_ARTIFACT_UUID = "5da04caf-9597-4760-aa71-b788eff1c03f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VnfExternalCp-template.yml,0,5da04caf-9597-4760-aa71-b788eff1c03f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=b1fced86-bac1-420f-b9b5-428bbf4d01eb TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Ended create Csar for component VNF External CP 2022-04-26T06:43:16.042Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "a47f71f3-17e4-41c0-97c5-acfc7e58d48b" INVARIANT_UUID = "cd9d9d77-294e-4c33-bc4b-9d028bdb4b0d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5da04caf-9597-4760-aa71-b788eff1c03f.assettoscacsar" CURR_ARTIFACT_UUID = "5da04caf-9597-4760-aa71-b788eff1c03f.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VnfExternalCp-csar.csar,0,5da04caf-9597-4760-aa71-b788eff1c03f.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=81ecbd67-7a8e-42a4-b6e9-09760a561f17 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:16.043Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "a47f71f3-17e4-41c0-97c5-acfc7e58d48b" INVARIANT_UUID = "cd9d9d77-294e-4c33-bc4b-9d028bdb4b0d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5da04caf-9597-4760-aa71-b788eff1c03f.assettoscacsar" CURR_ARTIFACT_UUID = "5da04caf-9597-4760-aa71-b788eff1c03f.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VnfExternalCp-csar.csar,0,5da04caf-9597-4760-aa71-b788eff1c03f.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=81ecbd67-7a8e-42a4-b6e9-09760a561f17 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:16.043Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "a47f71f3-17e4-41c0-97c5-acfc7e58d48b" INVARIANT_UUID = "cd9d9d77-294e-4c33-bc4b-9d028bdb4b0d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5da04caf-9597-4760-aa71-b788eff1c03f.assettoscacsar" CURR_ARTIFACT_UUID = "5da04caf-9597-4760-aa71-b788eff1c03f.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VnfExternalCp-csar.csar,0,5da04caf-9597-4760-aa71-b788eff1c03f.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=81ecbd67-7a8e-42a4-b6e9-09760a561f17 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:16.043Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "a47f71f3-17e4-41c0-97c5-acfc7e58d48b" INVARIANT_UUID = "cd9d9d77-294e-4c33-bc4b-9d028bdb4b0d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5da04caf-9597-4760-aa71-b788eff1c03f.assettoscacsar" CURR_ARTIFACT_UUID = "5da04caf-9597-4760-aa71-b788eff1c03f.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VnfExternalCp-csar.csar,0,5da04caf-9597-4760-aa71-b788eff1c03f.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=81ecbd67-7a8e-42a4-b6e9-09760a561f17 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:16.078Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VnfVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "85a5eeca-1b2a-483b-90ea-11c54d046054" INVARIANT_UUID = "ca97d76e-3707-40dc-a4dc-b42c89a409c8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VnfVirtualLink" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=85a5eeca-1b2a-483b-90ea-11c54d046054 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:43:16.079Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VnfVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "85a5eeca-1b2a-483b-90ea-11c54d046054" INVARIANT_UUID = "ca97d76e-3707-40dc-a4dc-b42c89a409c8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VnfVirtualLink" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=85a5eeca-1b2a-483b-90ea-11c54d046054 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:43:16.100Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VnfVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "85a5eeca-1b2a-483b-90ea-11c54d046054" INVARIANT_UUID = "ca97d76e-3707-40dc-a4dc-b42c89a409c8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "eef13708-3e55-4e5c-b1e3-2a331574c733.assettoscatemplate" CURR_ARTIFACT_UUID = "eef13708-3e55-4e5c-b1e3-2a331574c733.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnfvirtuallink-template.yml,0,eef13708-3e55-4e5c-b1e3-2a331574c733.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=6d5abd45-20f4-4be4-b233-aea1ba3ed051 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Starting to create Csar for component VnfVirtualLink 2022-04-26T06:43:16.117Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VnfVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "85a5eeca-1b2a-483b-90ea-11c54d046054" INVARIANT_UUID = "ca97d76e-3707-40dc-a4dc-b42c89a409c8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "eef13708-3e55-4e5c-b1e3-2a331574c733.assettoscatemplate" CURR_ARTIFACT_UUID = "eef13708-3e55-4e5c-b1e3-2a331574c733.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnfvirtuallink-template.yml,0,eef13708-3e55-4e5c-b1e3-2a331574c733.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=6d5abd45-20f4-4be4-b233-aea1ba3ed051 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Ended create Csar for component VnfVirtualLink 2022-04-26T06:43:16.157Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VnfVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "85a5eeca-1b2a-483b-90ea-11c54d046054" INVARIANT_UUID = "ca97d76e-3707-40dc-a4dc-b42c89a409c8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "eef13708-3e55-4e5c-b1e3-2a331574c733.assettoscacsar" CURR_ARTIFACT_UUID = "eef13708-3e55-4e5c-b1e3-2a331574c733.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Vnfvirtuallink-csar.csar,0,eef13708-3e55-4e5c-b1e3-2a331574c733.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=99ce025d-d455-4ac2-b05b-a42b0e052dda TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:16.158Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VnfVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "85a5eeca-1b2a-483b-90ea-11c54d046054" INVARIANT_UUID = "ca97d76e-3707-40dc-a4dc-b42c89a409c8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "eef13708-3e55-4e5c-b1e3-2a331574c733.assettoscacsar" CURR_ARTIFACT_UUID = "eef13708-3e55-4e5c-b1e3-2a331574c733.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Vnfvirtuallink-csar.csar,0,eef13708-3e55-4e5c-b1e3-2a331574c733.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=99ce025d-d455-4ac2-b05b-a42b0e052dda TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:16.158Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VnfVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "85a5eeca-1b2a-483b-90ea-11c54d046054" INVARIANT_UUID = "ca97d76e-3707-40dc-a4dc-b42c89a409c8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "eef13708-3e55-4e5c-b1e3-2a331574c733.assettoscacsar" CURR_ARTIFACT_UUID = "eef13708-3e55-4e5c-b1e3-2a331574c733.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Vnfvirtuallink-csar.csar,0,eef13708-3e55-4e5c-b1e3-2a331574c733.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=99ce025d-d455-4ac2-b05b-a42b0e052dda TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:16.158Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VnfVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "85a5eeca-1b2a-483b-90ea-11c54d046054" INVARIANT_UUID = "ca97d76e-3707-40dc-a4dc-b42c89a409c8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "eef13708-3e55-4e5c-b1e3-2a331574c733.assettoscacsar" CURR_ARTIFACT_UUID = "eef13708-3e55-4e5c-b1e3-2a331574c733.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Vnfvirtuallink-csar.csar,0,eef13708-3e55-4e5c-b1e3-2a331574c733.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=99ce025d-d455-4ac2-b05b-a42b0e052dda TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:16.200Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "b205ba80-2311-46b9-a820-1cb5860549c7" INVARIANT_UUID = "3b978d98-974a-4ac7-a31e-8aa2d269da1a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualObjectStorage" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=b205ba80-2311-46b9-a820-1cb5860549c7 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:43:16.202Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "b205ba80-2311-46b9-a820-1cb5860549c7" INVARIANT_UUID = "3b978d98-974a-4ac7-a31e-8aa2d269da1a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualObjectStorage" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=b205ba80-2311-46b9-a820-1cb5860549c7 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:43:16.231Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "b205ba80-2311-46b9-a820-1cb5860549c7" INVARIANT_UUID = "3b978d98-974a-4ac7-a31e-8aa2d269da1a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "df19acc2-0fbc-4d20-ba98-19f30f0a9a5e.assettoscatemplate" CURR_ARTIFACT_UUID = "df19acc2-0fbc-4d20-ba98-19f30f0a9a5e.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualobjectstorage-template.yml,0,df19acc2-0fbc-4d20-ba98-19f30f0a9a5e.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=c1316da1-3692-435f-a20f-2fa76ee6d51e TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Starting to create Csar for component VDU VirtualObjectStorage 2022-04-26T06:43:16.263Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "b205ba80-2311-46b9-a820-1cb5860549c7" INVARIANT_UUID = "3b978d98-974a-4ac7-a31e-8aa2d269da1a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "df19acc2-0fbc-4d20-ba98-19f30f0a9a5e.assettoscatemplate" CURR_ARTIFACT_UUID = "df19acc2-0fbc-4d20-ba98-19f30f0a9a5e.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualobjectstorage-template.yml,0,df19acc2-0fbc-4d20-ba98-19f30f0a9a5e.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=c1316da1-3692-435f-a20f-2fa76ee6d51e TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Ended create Csar for component VDU VirtualObjectStorage 2022-04-26T06:43:16.351Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "b205ba80-2311-46b9-a820-1cb5860549c7" INVARIANT_UUID = "3b978d98-974a-4ac7-a31e-8aa2d269da1a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "df19acc2-0fbc-4d20-ba98-19f30f0a9a5e.assettoscacsar" CURR_ARTIFACT_UUID = "df19acc2-0fbc-4d20-ba98-19f30f0a9a5e.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduVirtualobjectstorage-csar.csar,0,df19acc2-0fbc-4d20-ba98-19f30f0a9a5e.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=af21d0a7-cba6-4847-8de5-fa8a0076e7a3 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:16.351Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "b205ba80-2311-46b9-a820-1cb5860549c7" INVARIANT_UUID = "3b978d98-974a-4ac7-a31e-8aa2d269da1a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "df19acc2-0fbc-4d20-ba98-19f30f0a9a5e.assettoscacsar" CURR_ARTIFACT_UUID = "df19acc2-0fbc-4d20-ba98-19f30f0a9a5e.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduVirtualobjectstorage-csar.csar,0,df19acc2-0fbc-4d20-ba98-19f30f0a9a5e.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=af21d0a7-cba6-4847-8de5-fa8a0076e7a3 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:16.352Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "b205ba80-2311-46b9-a820-1cb5860549c7" INVARIANT_UUID = "3b978d98-974a-4ac7-a31e-8aa2d269da1a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "df19acc2-0fbc-4d20-ba98-19f30f0a9a5e.assettoscacsar" CURR_ARTIFACT_UUID = "df19acc2-0fbc-4d20-ba98-19f30f0a9a5e.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduVirtualobjectstorage-csar.csar,0,df19acc2-0fbc-4d20-ba98-19f30f0a9a5e.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=af21d0a7-cba6-4847-8de5-fa8a0076e7a3 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:16.352Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "b205ba80-2311-46b9-a820-1cb5860549c7" INVARIANT_UUID = "3b978d98-974a-4ac7-a31e-8aa2d269da1a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "df19acc2-0fbc-4d20-ba98-19f30f0a9a5e.assettoscacsar" CURR_ARTIFACT_UUID = "df19acc2-0fbc-4d20-ba98-19f30f0a9a5e.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduVirtualobjectstorage-csar.csar,0,df19acc2-0fbc-4d20-ba98-19f30f0a9a5e.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=af21d0a7-cba6-4847-8de5-fa8a0076e7a3 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:16.437Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "395f8709-5fee-411a-b94f-381ae6e11c2a" INVARIANT_UUID = "4437ec39-67f2-453c-98a3-4fa4f2e746ad" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualFileStorage" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=395f8709-5fee-411a-b94f-381ae6e11c2a TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:43:16.437Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "395f8709-5fee-411a-b94f-381ae6e11c2a" INVARIANT_UUID = "4437ec39-67f2-453c-98a3-4fa4f2e746ad" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualFileStorage" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=395f8709-5fee-411a-b94f-381ae6e11c2a TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:43:16.458Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "395f8709-5fee-411a-b94f-381ae6e11c2a" INVARIANT_UUID = "4437ec39-67f2-453c-98a3-4fa4f2e746ad" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "24fc8a9e-9810-4a6c-bfa3-6a6e667b77d5.assettoscatemplate" CURR_ARTIFACT_UUID = "24fc8a9e-9810-4a6c-bfa3-6a6e667b77d5.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualfilestorage-template.yml,0,24fc8a9e-9810-4a6c-bfa3-6a6e667b77d5.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=e748b9f5-0ff9-42d5-9901-20e0d61e875a TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Starting to create Csar for component VDU VirtualFileStorage 2022-04-26T06:43:16.534Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "395f8709-5fee-411a-b94f-381ae6e11c2a" INVARIANT_UUID = "4437ec39-67f2-453c-98a3-4fa4f2e746ad" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "24fc8a9e-9810-4a6c-bfa3-6a6e667b77d5.assettoscatemplate" CURR_ARTIFACT_UUID = "24fc8a9e-9810-4a6c-bfa3-6a6e667b77d5.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualfilestorage-template.yml,0,24fc8a9e-9810-4a6c-bfa3-6a6e667b77d5.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=e748b9f5-0ff9-42d5-9901-20e0d61e875a TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Ended create Csar for component VDU VirtualFileStorage 2022-04-26T06:43:16.581Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "395f8709-5fee-411a-b94f-381ae6e11c2a" INVARIANT_UUID = "4437ec39-67f2-453c-98a3-4fa4f2e746ad" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "24fc8a9e-9810-4a6c-bfa3-6a6e667b77d5.assettoscacsar" CURR_ARTIFACT_UUID = "24fc8a9e-9810-4a6c-bfa3-6a6e667b77d5.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduVirtualfilestorage-csar.csar,0,24fc8a9e-9810-4a6c-bfa3-6a6e667b77d5.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=d8a5ca15-248d-49c2-b42f-4228f3c79f9c TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:16.581Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "395f8709-5fee-411a-b94f-381ae6e11c2a" INVARIANT_UUID = "4437ec39-67f2-453c-98a3-4fa4f2e746ad" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "24fc8a9e-9810-4a6c-bfa3-6a6e667b77d5.assettoscacsar" CURR_ARTIFACT_UUID = "24fc8a9e-9810-4a6c-bfa3-6a6e667b77d5.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduVirtualfilestorage-csar.csar,0,24fc8a9e-9810-4a6c-bfa3-6a6e667b77d5.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=d8a5ca15-248d-49c2-b42f-4228f3c79f9c TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:16.581Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "395f8709-5fee-411a-b94f-381ae6e11c2a" INVARIANT_UUID = "4437ec39-67f2-453c-98a3-4fa4f2e746ad" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "24fc8a9e-9810-4a6c-bfa3-6a6e667b77d5.assettoscacsar" CURR_ARTIFACT_UUID = "24fc8a9e-9810-4a6c-bfa3-6a6e667b77d5.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduVirtualfilestorage-csar.csar,0,24fc8a9e-9810-4a6c-bfa3-6a6e667b77d5.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=d8a5ca15-248d-49c2-b42f-4228f3c79f9c TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:16.581Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "395f8709-5fee-411a-b94f-381ae6e11c2a" INVARIANT_UUID = "4437ec39-67f2-453c-98a3-4fa4f2e746ad" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "24fc8a9e-9810-4a6c-bfa3-6a6e667b77d5.assettoscacsar" CURR_ARTIFACT_UUID = "24fc8a9e-9810-4a6c-bfa3-6a6e667b77d5.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduVirtualfilestorage-csar.csar,0,24fc8a9e-9810-4a6c-bfa3-6a6e667b77d5.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=d8a5ca15-248d-49c2-b42f-4228f3c79f9c TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:16.660Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "8a8dae00-7a2e-4c88-8748-6c7e229f9eb1" INVARIANT_UUID = "4fc68634-baee-4542-9e4b-a97e90e15472" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualBlockStorage" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=8a8dae00-7a2e-4c88-8748-6c7e229f9eb1 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:43:16.661Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "8a8dae00-7a2e-4c88-8748-6c7e229f9eb1" INVARIANT_UUID = "4fc68634-baee-4542-9e4b-a97e90e15472" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualBlockStorage" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=8a8dae00-7a2e-4c88-8748-6c7e229f9eb1 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:43:16.732Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "8a8dae00-7a2e-4c88-8748-6c7e229f9eb1" INVARIANT_UUID = "4fc68634-baee-4542-9e4b-a97e90e15472" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a91793cd-24e8-45ac-8c4c-50356115a15f.assettoscatemplate" CURR_ARTIFACT_UUID = "a91793cd-24e8-45ac-8c4c-50356115a15f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualblockstorage-template.yml,0,a91793cd-24e8-45ac-8c4c-50356115a15f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=bf4636d2-a8e0-4cb9-8c7f-0b9c3b798d68 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Starting to create Csar for component VDU VirtualBlockStorage 2022-04-26T06:43:16.751Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "8a8dae00-7a2e-4c88-8748-6c7e229f9eb1" INVARIANT_UUID = "4fc68634-baee-4542-9e4b-a97e90e15472" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a91793cd-24e8-45ac-8c4c-50356115a15f.assettoscatemplate" CURR_ARTIFACT_UUID = "a91793cd-24e8-45ac-8c4c-50356115a15f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualblockstorage-template.yml,0,a91793cd-24e8-45ac-8c4c-50356115a15f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=bf4636d2-a8e0-4cb9-8c7f-0b9c3b798d68 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Ended create Csar for component VDU VirtualBlockStorage 2022-04-26T06:43:16.849Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "8a8dae00-7a2e-4c88-8748-6c7e229f9eb1" INVARIANT_UUID = "4fc68634-baee-4542-9e4b-a97e90e15472" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a91793cd-24e8-45ac-8c4c-50356115a15f.assettoscacsar" CURR_ARTIFACT_UUID = "a91793cd-24e8-45ac-8c4c-50356115a15f.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduVirtualblockstorage-csar.csar,0,a91793cd-24e8-45ac-8c4c-50356115a15f.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=39f95ebb-4abb-43f7-a478-e078322aca06 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:16.849Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "8a8dae00-7a2e-4c88-8748-6c7e229f9eb1" INVARIANT_UUID = "4fc68634-baee-4542-9e4b-a97e90e15472" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a91793cd-24e8-45ac-8c4c-50356115a15f.assettoscacsar" CURR_ARTIFACT_UUID = "a91793cd-24e8-45ac-8c4c-50356115a15f.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduVirtualblockstorage-csar.csar,0,a91793cd-24e8-45ac-8c4c-50356115a15f.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=39f95ebb-4abb-43f7-a478-e078322aca06 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:16.849Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "8a8dae00-7a2e-4c88-8748-6c7e229f9eb1" INVARIANT_UUID = "4fc68634-baee-4542-9e4b-a97e90e15472" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a91793cd-24e8-45ac-8c4c-50356115a15f.assettoscacsar" CURR_ARTIFACT_UUID = "a91793cd-24e8-45ac-8c4c-50356115a15f.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduVirtualblockstorage-csar.csar,0,a91793cd-24e8-45ac-8c4c-50356115a15f.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=39f95ebb-4abb-43f7-a478-e078322aca06 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:16.849Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "8a8dae00-7a2e-4c88-8748-6c7e229f9eb1" INVARIANT_UUID = "4fc68634-baee-4542-9e4b-a97e90e15472" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a91793cd-24e8-45ac-8c4c-50356115a15f.assettoscacsar" CURR_ARTIFACT_UUID = "a91793cd-24e8-45ac-8c4c-50356115a15f.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduVirtualblockstorage-csar.csar,0,a91793cd-24e8-45ac-8c4c-50356115a15f.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=39f95ebb-4abb-43f7-a478-e078322aca06 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:16.888Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NsVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "29514875-e23e-424a-a42f-aaa4b89db370" INVARIANT_UUID = "27a7c46e-f94a-4333-9842-9f54fee7248a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NsVirtualLink" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=29514875-e23e-424a-a42f-aaa4b89db370 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:43:16.889Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NsVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "29514875-e23e-424a-a42f-aaa4b89db370" INVARIANT_UUID = "27a7c46e-f94a-4333-9842-9f54fee7248a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NsVirtualLink" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=29514875-e23e-424a-a42f-aaa4b89db370 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:43:16.914Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NsVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "29514875-e23e-424a-a42f-aaa4b89db370" INVARIANT_UUID = "27a7c46e-f94a-4333-9842-9f54fee7248a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscatemplate" CURR_ARTIFACT_UUID = "a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nsvirtuallink-template.yml,0,a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=63a9ca58-5b25-46de-b8b9-68b5547b3aba TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Starting to create Csar for component NsVirtualLink 2022-04-26T06:43:16.931Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NsVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "29514875-e23e-424a-a42f-aaa4b89db370" INVARIANT_UUID = "27a7c46e-f94a-4333-9842-9f54fee7248a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscatemplate" CURR_ARTIFACT_UUID = "a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nsvirtuallink-template.yml,0,a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=63a9ca58-5b25-46de-b8b9-68b5547b3aba TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Ended create Csar for component NsVirtualLink 2022-04-26T06:43:16.972Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NsVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "29514875-e23e-424a-a42f-aaa4b89db370" INVARIANT_UUID = "27a7c46e-f94a-4333-9842-9f54fee7248a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscacsar" CURR_ARTIFACT_UUID = "a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Nsvirtuallink-csar.csar,0,a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=d6f92f6b-db33-42d8-8968-3e151219176f TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:16.972Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NsVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "29514875-e23e-424a-a42f-aaa4b89db370" INVARIANT_UUID = "27a7c46e-f94a-4333-9842-9f54fee7248a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscacsar" CURR_ARTIFACT_UUID = "a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Nsvirtuallink-csar.csar,0,a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=d6f92f6b-db33-42d8-8968-3e151219176f TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:16.972Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NsVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "29514875-e23e-424a-a42f-aaa4b89db370" INVARIANT_UUID = "27a7c46e-f94a-4333-9842-9f54fee7248a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscacsar" CURR_ARTIFACT_UUID = "a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Nsvirtuallink-csar.csar,0,a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=d6f92f6b-db33-42d8-8968-3e151219176f TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:16.972Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NsVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "29514875-e23e-424a-a42f-aaa4b89db370" INVARIANT_UUID = "27a7c46e-f94a-4333-9842-9f54fee7248a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscacsar" CURR_ARTIFACT_UUID = "a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Nsvirtuallink-csar.csar,0,a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=d6f92f6b-db33-42d8-8968-3e151219176f TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:16.975Z [qtp2085002312-17] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NsVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "29514875-e23e-424a-a42f-aaa4b89db370" INVARIANT_UUID = "27a7c46e-f94a-4333-9842-9f54fee7248a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscacsar" CURR_ARTIFACT_UUID = "a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Nsvirtuallink-csar.csar,0,a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Create Resource - validateLifecycleTypesCreate ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=d6f92f6b-db33-42d8-8968-3e151219176f TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=400 Interface tosca.interfaces.nfv.nslcm required is missing 2022-04-26T06:43:16.975Z [qtp2085002312-17] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NsVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "29514875-e23e-424a-a42f-aaa4b89db370" INVARIANT_UUID = "27a7c46e-f94a-4333-9842-9f54fee7248a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscacsar" CURR_ARTIFACT_UUID = "a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Nsvirtuallink-csar.csar,0,a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=Create Resource - validateLifecycleTypesCreate ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=d6f92f6b-db33-42d8-8968-3e151219176f TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Operation towards database failed. 2022-04-26T06:43:16.975Z [qtp2085002312-17] WARN o.o.s.e.AbstractSdncException AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NsVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "29514875-e23e-424a-a42f-aaa4b89db370" INVARIANT_UUID = "27a7c46e-f94a-4333-9842-9f54fee7248a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscacsar" CURR_ARTIFACT_UUID = "a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Nsvirtuallink-csar.csar,0,a695a087-3bae-4981-b0f8-fe413a45fe3d.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=d6f92f6b-db33-42d8-8968-3e151219176f TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=900 2022-04-26T06:43:17.020Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e3243724-47fc-4f43-a658-0f6a682dc398" INVARIANT_UUID = "604e3d90-f244-43f3-b2b4-59f93a7833b5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NS" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=e3243724-47fc-4f43-a658-0f6a682dc398 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=900 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:43:17.021Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e3243724-47fc-4f43-a658-0f6a682dc398" INVARIANT_UUID = "604e3d90-f244-43f3-b2b4-59f93a7833b5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NS" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=e3243724-47fc-4f43-a658-0f6a682dc398 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:43:17.047Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e3243724-47fc-4f43-a658-0f6a682dc398" INVARIANT_UUID = "604e3d90-f244-43f3-b2b4-59f93a7833b5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "7938ae38-61b4-43f6-800b-9eb5eeb2500a.assettoscatemplate" CURR_ARTIFACT_UUID = "7938ae38-61b4-43f6-800b-9eb5eeb2500a.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ns-template.yml,0,7938ae38-61b4-43f6-800b-9eb5eeb2500a.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=52349b27-5375-4fd2-b918-f1afaa6c6d29 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Starting to create Csar for component NS 2022-04-26T06:43:17.067Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e3243724-47fc-4f43-a658-0f6a682dc398" INVARIANT_UUID = "604e3d90-f244-43f3-b2b4-59f93a7833b5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "7938ae38-61b4-43f6-800b-9eb5eeb2500a.assettoscatemplate" CURR_ARTIFACT_UUID = "7938ae38-61b4-43f6-800b-9eb5eeb2500a.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ns-template.yml,0,7938ae38-61b4-43f6-800b-9eb5eeb2500a.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=52349b27-5375-4fd2-b918-f1afaa6c6d29 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Ended create Csar for component NS 2022-04-26T06:43:17.135Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e3243724-47fc-4f43-a658-0f6a682dc398" INVARIANT_UUID = "604e3d90-f244-43f3-b2b4-59f93a7833b5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "7938ae38-61b4-43f6-800b-9eb5eeb2500a.assettoscacsar" CURR_ARTIFACT_UUID = "7938ae38-61b4-43f6-800b-9eb5eeb2500a.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Ns-csar.csar,0,7938ae38-61b4-43f6-800b-9eb5eeb2500a.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=560a4aff-aae6-4d0f-8bf6-c4ab02f785d8 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:17.135Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e3243724-47fc-4f43-a658-0f6a682dc398" INVARIANT_UUID = "604e3d90-f244-43f3-b2b4-59f93a7833b5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "7938ae38-61b4-43f6-800b-9eb5eeb2500a.assettoscacsar" CURR_ARTIFACT_UUID = "7938ae38-61b4-43f6-800b-9eb5eeb2500a.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Ns-csar.csar,0,7938ae38-61b4-43f6-800b-9eb5eeb2500a.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=560a4aff-aae6-4d0f-8bf6-c4ab02f785d8 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:17.135Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e3243724-47fc-4f43-a658-0f6a682dc398" INVARIANT_UUID = "604e3d90-f244-43f3-b2b4-59f93a7833b5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "7938ae38-61b4-43f6-800b-9eb5eeb2500a.assettoscacsar" CURR_ARTIFACT_UUID = "7938ae38-61b4-43f6-800b-9eb5eeb2500a.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Ns-csar.csar,0,7938ae38-61b4-43f6-800b-9eb5eeb2500a.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=560a4aff-aae6-4d0f-8bf6-c4ab02f785d8 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:17.136Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "e3243724-47fc-4f43-a658-0f6a682dc398" INVARIANT_UUID = "604e3d90-f244-43f3-b2b4-59f93a7833b5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "7938ae38-61b4-43f6-800b-9eb5eeb2500a.assettoscacsar" CURR_ARTIFACT_UUID = "7938ae38-61b4-43f6-800b-9eb5eeb2500a.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Ns-csar.csar,0,7938ae38-61b4-43f6-800b-9eb5eeb2500a.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=560a4aff-aae6-4d0f-8bf6-c4ab02f785d8 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:17.163Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f9e60d3e-f10c-42a5-b1ff-fc410f73aade ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:43:17.163Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f9e60d3e-f10c-42a5-b1ff-fc410f73aade ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:43:17.164Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=f9e60d3e-f10c-42a5-b1ff-fc410f73aade ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:43:17.164Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=f9e60d3e-f10c-42a5-b1ff-fc410f73aade ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:43:17.245Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "fc930f19-dacb-4a91-b4f6-ff5ab5ccb804" INVARIANT_UUID = "81ef174f-0b83-4140-8c2b-496b47207bcf" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.Compute" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=fc930f19-dacb-4a91-b4f6-ff5ab5ccb804 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:43:17.245Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "fc930f19-dacb-4a91-b4f6-ff5ab5ccb804" INVARIANT_UUID = "81ef174f-0b83-4140-8c2b-496b47207bcf" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.Compute" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=fc930f19-dacb-4a91-b4f6-ff5ab5ccb804 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:43:17.272Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "fc930f19-dacb-4a91-b4f6-ff5ab5ccb804" INVARIANT_UUID = "81ef174f-0b83-4140-8c2b-496b47207bcf" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2f618a0d-10d2-4701-a6fe-f8f722af5d0f.assettoscatemplate" CURR_ARTIFACT_UUID = "2f618a0d-10d2-4701-a6fe-f8f722af5d0f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCompute-template.yml,0,2f618a0d-10d2-4701-a6fe-f8f722af5d0f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=c65b5665-c84e-441e-aa96-616af03b6c61 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Starting to create Csar for component VDU Compute 2022-04-26T06:43:17.347Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "fc930f19-dacb-4a91-b4f6-ff5ab5ccb804" INVARIANT_UUID = "81ef174f-0b83-4140-8c2b-496b47207bcf" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2f618a0d-10d2-4701-a6fe-f8f722af5d0f.assettoscatemplate" CURR_ARTIFACT_UUID = "2f618a0d-10d2-4701-a6fe-f8f722af5d0f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCompute-template.yml,0,2f618a0d-10d2-4701-a6fe-f8f722af5d0f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=c65b5665-c84e-441e-aa96-616af03b6c61 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Ended create Csar for component VDU Compute 2022-04-26T06:43:17.364Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18525f1e-d99b-4241-9764-15b19d497729 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:43:17.364Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18525f1e-d99b-4241-9764-15b19d497729 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8da24af 2022-04-26T06:43:17.365Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18525f1e-d99b-4241-9764-15b19d497729 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:43:17.365Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18525f1e-d99b-4241-9764-15b19d497729 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:43:17.396Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "fc930f19-dacb-4a91-b4f6-ff5ab5ccb804" INVARIANT_UUID = "81ef174f-0b83-4140-8c2b-496b47207bcf" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2f618a0d-10d2-4701-a6fe-f8f722af5d0f.assettoscacsar" CURR_ARTIFACT_UUID = "2f618a0d-10d2-4701-a6fe-f8f722af5d0f.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduCompute-csar.csar,0,2f618a0d-10d2-4701-a6fe-f8f722af5d0f.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=4a3b7bc4-cf9a-4aeb-b80a-15f1388ba669 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:17.397Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "fc930f19-dacb-4a91-b4f6-ff5ab5ccb804" INVARIANT_UUID = "81ef174f-0b83-4140-8c2b-496b47207bcf" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2f618a0d-10d2-4701-a6fe-f8f722af5d0f.assettoscacsar" CURR_ARTIFACT_UUID = "2f618a0d-10d2-4701-a6fe-f8f722af5d0f.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduCompute-csar.csar,0,2f618a0d-10d2-4701-a6fe-f8f722af5d0f.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=4a3b7bc4-cf9a-4aeb-b80a-15f1388ba669 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:17.397Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "fc930f19-dacb-4a91-b4f6-ff5ab5ccb804" INVARIANT_UUID = "81ef174f-0b83-4140-8c2b-496b47207bcf" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2f618a0d-10d2-4701-a6fe-f8f722af5d0f.assettoscacsar" CURR_ARTIFACT_UUID = "2f618a0d-10d2-4701-a6fe-f8f722af5d0f.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduCompute-csar.csar,0,2f618a0d-10d2-4701-a6fe-f8f722af5d0f.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=4a3b7bc4-cf9a-4aeb-b80a-15f1388ba669 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:17.397Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "fc930f19-dacb-4a91-b4f6-ff5ab5ccb804" INVARIANT_UUID = "81ef174f-0b83-4140-8c2b-496b47207bcf" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2f618a0d-10d2-4701-a6fe-f8f722af5d0f.assettoscacsar" CURR_ARTIFACT_UUID = "2f618a0d-10d2-4701-a6fe-f8f722af5d0f.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduCompute-csar.csar,0,2f618a0d-10d2-4701-a6fe-f8f722af5d0f.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=4a3b7bc4-cf9a-4aeb-b80a-15f1388ba669 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:17.445Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Sap" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "545b3a89-5e16-47cd-b051-89916b47875f" INVARIANT_UUID = "a6ed195d-9a5c-44f8-8fbe-adeeb4813494" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Sap" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=545b3a89-5e16-47cd-b051-89916b47875f TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:43:17.446Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Sap" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "545b3a89-5e16-47cd-b051-89916b47875f" INVARIANT_UUID = "a6ed195d-9a5c-44f8-8fbe-adeeb4813494" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Sap" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=545b3a89-5e16-47cd-b051-89916b47875f TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:43:17.466Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Sap" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "545b3a89-5e16-47cd-b051-89916b47875f" INVARIANT_UUID = "a6ed195d-9a5c-44f8-8fbe-adeeb4813494" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "4d42bacd-97df-4daf-8429-c3ea0adc6786.assettoscatemplate" CURR_ARTIFACT_UUID = "4d42bacd-97df-4daf-8429-c3ea0adc6786.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Sap-template.yml,0,4d42bacd-97df-4daf-8429-c3ea0adc6786.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=1f622b1d-3cb5-4d31-aa89-f225b6ff948e TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Starting to create Csar for component Sap 2022-04-26T06:43:17.484Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Sap" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "545b3a89-5e16-47cd-b051-89916b47875f" INVARIANT_UUID = "a6ed195d-9a5c-44f8-8fbe-adeeb4813494" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "4d42bacd-97df-4daf-8429-c3ea0adc6786.assettoscatemplate" CURR_ARTIFACT_UUID = "4d42bacd-97df-4daf-8429-c3ea0adc6786.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Sap-template.yml,0,4d42bacd-97df-4daf-8429-c3ea0adc6786.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=1f622b1d-3cb5-4d31-aa89-f225b6ff948e TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Ended create Csar for component Sap 2022-04-26T06:43:17.523Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Sap" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "545b3a89-5e16-47cd-b051-89916b47875f" INVARIANT_UUID = "a6ed195d-9a5c-44f8-8fbe-adeeb4813494" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "4d42bacd-97df-4daf-8429-c3ea0adc6786.assettoscacsar" CURR_ARTIFACT_UUID = "4d42bacd-97df-4daf-8429-c3ea0adc6786.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Sap-csar.csar,0,4d42bacd-97df-4daf-8429-c3ea0adc6786.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=380e7039-db4b-4513-9855-e7e4302a0f9b TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 LifecycleState cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:17.523Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Sap" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "545b3a89-5e16-47cd-b051-89916b47875f" INVARIANT_UUID = "a6ed195d-9a5c-44f8-8fbe-adeeb4813494" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "4d42bacd-97df-4daf-8429-c3ea0adc6786.assettoscacsar" CURR_ARTIFACT_UUID = "4d42bacd-97df-4daf-8429-c3ea0adc6786.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Sap-csar.csar,0,4d42bacd-97df-4daf-8429-c3ea0adc6786.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=380e7039-db4b-4513-9855-e7e4302a0f9b TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:17.523Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Sap" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "545b3a89-5e16-47cd-b051-89916b47875f" INVARIANT_UUID = "a6ed195d-9a5c-44f8-8fbe-adeeb4813494" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "4d42bacd-97df-4daf-8429-c3ea0adc6786.assettoscacsar" CURR_ARTIFACT_UUID = "4d42bacd-97df-4daf-8429-c3ea0adc6786.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Sap-csar.csar,0,4d42bacd-97df-4daf-8429-c3ea0adc6786.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=380e7039-db4b-4513-9855-e7e4302a0f9b TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Creator cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:17.523Z [qtp2085002312-17] INFO o.o.s.b.c.i.ComponentBusinessLogic AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Sap" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "545b3a89-5e16-47cd-b051-89916b47875f" INVARIANT_UUID = "a6ed195d-9a5c-44f8-8fbe-adeeb4813494" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "4d42bacd-97df-4daf-8429-c3ea0adc6786.assettoscacsar" CURR_ARTIFACT_UUID = "4d42bacd-97df-4daf-8429-c3ea0adc6786.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Sap-csar.csar,0,4d42bacd-97df-4daf-8429-c3ea0adc6786.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=380e7039-db4b-4513-9855-e7e4302a0f9b TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Is Highest Version cannot be defined by user. This field will be overridden by the application 2022-04-26T06:43:17.572Z [qtp2085002312-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "9190d2dd-fbaf-4448-957f-ddd15941614d" INVARIANT_UUID = "c545d4e4-4411-4783-8b71-8643cc69288c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VduCp" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=9190d2dd-fbaf-4448-957f-ddd15941614d TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1650953634, consumerId=sdc-AUTO1-1650953634, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2022-04-26T06:43:17.573Z [qtp2085002312-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "9190d2dd-fbaf-4448-957f-ddd15941614d" INVARIANT_UUID = "c545d4e4-4411-4783-8b71-8643cc69288c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VduCp" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=9190d2dd-fbaf-4448-957f-ddd15941614d TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2022-04-26T06:43:17.633Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "9190d2dd-fbaf-4448-957f-ddd15941614d" INVARIANT_UUID = "c545d4e4-4411-4783-8b71-8643cc69288c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "517b9891-9cfe-4f2b-ad11-f850bbb6f773.assettoscatemplate" CURR_ARTIFACT_UUID = "517b9891-9cfe-4f2b-ad11-f850bbb6f773.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCp-template.yml,0,517b9891-9cfe-4f2b-ad11-f850bbb6f773.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=db9bcc54-1acd-434c-a252-748365ce5f46 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Starting to create Csar for component VDU Cp 2022-04-26T06:43:17.735Z [qtp2085002312-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "9190d2dd-fbaf-4448-957f-ddd15941614d" INVARIANT_UUID = "c545d4e4-4411-4783-8b71-8643cc69288c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "517b9891-9cfe-4f2b-ad11-f850bbb6f773.assettoscatemplate" CURR_ARTIFACT_UUID = "517b9891-9cfe-4f2b-ad11-f850bbb6f773.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCp-template.yml,0,517b9891-9cfe-4f2b-ad11-f850bbb6f773.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=db9bcc54-1acd-434c-a252-748365ce5f46 TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 Ended create Csar for component VDU Cp 2022-04-26T06:43:20.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:43:20.382Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:43:20.401Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=18525f1e-d99b-4241-9764-15b19d497729 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:43:20.401Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=18525f1e-d99b-4241-9764-15b19d497729 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:43:20.922Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5600879c-4370-4b1c-b63d-7fec6a4d7ea8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:43:20.922Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5600879c-4370-4b1c-b63d-7fec6a4d7ea8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3cb69051 2022-04-26T06:43:20.922Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5600879c-4370-4b1c-b63d-7fec6a4d7ea8 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:43:20.922Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5600879c-4370-4b1c-b63d-7fec6a4d7ea8 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:43:22.324Z [qtp2085002312-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "9190d2dd-fbaf-4448-957f-ddd15941614d" INVARIANT_UUID = "c545d4e4-4411-4783-8b71-8643cc69288c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "517b9891-9cfe-4f2b-ad11-f850bbb6f773.assettoscacsar" CURR_ARTIFACT_UUID = "517b9891-9cfe-4f2b-ad11-f850bbb6f773.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduCp-csar.csar,0,517b9891-9cfe-4f2b-ad11-f850bbb6f773.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=10877 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/upload/resource/import ServiceInstanceID=b16f656a-cab0-4707-a93a-8f6e723e6a4f TargetServiceName=null EntryTimestamp=2022-04-26T06:43:11.447Z InvocationID=a05c57a5-33c5-4aea-aec2-2c0188d1fca2 ErrorCode=500 LogTimestamp=2022-04-26T06:43:22.324289Z ResponseDesc=Created Exiting. 2022-04-26T06:43:22.325Z [qtp2085002312-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/resource/import HTTP/1.1 SC="201" 2022-04-26T06:43:22.336Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ff0bd73e-a6ba-401e-9cbb-21300493d847 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServiceName=v1/catalog/uploadType/grouptypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:43:22.336269Z InvocationID=a0df7b5d-7e2d-4bba-b653-e6ae4da0220d LogTimestamp=2022-04-26T06:43:22.336361Z Entering 2022-04-26T06:43:22.336Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ff0bd73e-a6ba-401e-9cbb-21300493d847 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServiceName=v1/catalog/uploadType/grouptypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:43:22.336269Z InvocationID=a0df7b5d-7e2d-4bba-b653-e6ae4da0220d LogTimestamp=2022-04-26T06:43:22.336361Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/catalog/uploadType/grouptypes HTTP/1.1 2022-04-26T06:43:22.420Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2022-04-26T06:43:22.337Z InvocationID=804ce62e-c0cd-4de0-9b48-4fb15741bf23 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send GROUP_TYPE : etsi sol001 v2.5.1.tosca.groups.root.1.0.grouptype to dao for create 2022-04-26T06:43:22.431Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GroupTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2022-04-26T06:43:22.337Z InvocationID=804ce62e-c0cd-4de0-9b48-4fb15741bf23 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=GroupType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:23.126Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2022-04-26T06:43:22.337Z InvocationID=804ce62e-c0cd-4de0-9b48-4fb15741bf23 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send GROUP_TYPE : etsi sol001 v2.5.1.tosca.groups.nfv.placementgroup.1.0.grouptype to dao for create 2022-04-26T06:43:23.223Z [qtp2085002312-20] INFO o.o.s.b.m.o.impl.GroupTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2022-04-26T06:43:22.337Z InvocationID=804ce62e-c0cd-4de0-9b48-4fb15741bf23 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=GroupType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:23.226Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2022-04-26T06:43:22.337Z InvocationID=804ce62e-c0cd-4de0-9b48-4fb15741bf23 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb all GroupTypeDefinition(super=GroupTypeDataDefinition [uniqueId=ETSI SOL001 v2.5.1.tosca.groups.nfv.PlacementGroup.1.0.grouptype, type=tosca.groups.nfv.PlacementGroup, name=NFV PlacementGroup, icon=group-nfv-placementgroup, version=1.0, members=[tosca.nodes.nfv.Vdu.Compute, tosca.nodes.nfv.VnfVirtualLink], metadata=null, description=PlacementGroup is used for describing the affinity or anti-affinity relationship applicable between the virtualization containers to be created based on different VDUs, or between internal VLs to be created based on different VnfVirtualLinkDesc(s), creationTime=1650955403131, modificationTime=1650955403131], properties=[PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=ETSI SOL001 v2.5.1.tosca.groups.nfv.PlacementGroup.1.0.grouptype.description, type=string, required=true, defaultValue=null, description=Human readable description of the group, schema=null, password=false, name=description, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, model=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=null, metadata=null), name=description, constraints=null]]], capabilities={}, model=ETSI SOL001 v2.5.1) were created successfully!!! 2022-04-26T06:43:23.959Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=5600879c-4370-4b1c-b63d-7fec6a4d7ea8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:43:23.960Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=5600879c-4370-4b1c-b63d-7fec6a4d7ea8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:43:24.020Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1682 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2022-04-26T06:43:22.337Z InvocationID=804ce62e-c0cd-4de0-9b48-4fb15741bf23 LogTimestamp=2022-04-26T06:43:24.019911Z ResponseDesc=Created Exiting. 2022-04-26T06:43:24.020Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/grouptypes HTTP/1.1 SC="201" 2022-04-26T06:43:24.027Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=fea0e4fb-9722-4edc-9851-2f4d76731552 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServiceName=v1/catalog/uploadType/policytypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:43:24.027712Z InvocationID=7d8242cb-2b5f-4e97-91d0-4f64988a6f77 LogTimestamp=2022-04-26T06:43:24.027746Z Entering 2022-04-26T06:43:24.028Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=fea0e4fb-9722-4edc-9851-2f4d76731552 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServiceName=v1/catalog/uploadType/policytypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:43:24.027712Z InvocationID=7d8242cb-2b5f-4e97-91d0-4f64988a6f77 LogTimestamp=2022-04-26T06:43:24.027746Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/catalog/uploadType/policytypes HTTP/1.1 2022-04-26T06:43:24.128Z [qtp2085002312-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : etsi sol001 v2.5.1.tosca.policies.root.1.0.policytype to dao for create 2022-04-26T06:43:24.223Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.PolicyTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=PolicyType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:24.226Z [qtp2085002312-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : etsi sol001 v2.5.1.tosca.policies.placement.1.0.policytype to dao for create 2022-04-26T06:43:24.318Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.PolicyTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=PolicyType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:24.326Z [qtp2085002312-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : etsi sol001 v2.5.1.tosca.policies.nfv.instantiationlevels.1.0.policytype to dao for create 2022-04-26T06:43:24.425Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.PolicyTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=PolicyType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:24.528Z [qtp2085002312-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : etsi sol001 v2.5.1.tosca.policies.nfv.vduinstantiationlevels.1.0.policytype to dao for create 2022-04-26T06:43:24.534Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.PolicyTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=PolicyType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:24.627Z [qtp2085002312-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : etsi sol001 v2.5.1.tosca.policies.nfv.virtuallinkinstantiationlevels.1.0.policytype to dao for create 2022-04-26T06:43:24.720Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.PolicyTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=PolicyType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:24.726Z [qtp2085002312-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : etsi sol001 v2.5.1.tosca.policies.nfv.scalingaspects.1.0.policytype to dao for create 2022-04-26T06:43:24.818Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.PolicyTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=PolicyType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:24.827Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dbd659ef-7499-47c2-ac32-417d92047f82 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:43:24.827Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dbd659ef-7499-47c2-ac32-417d92047f82 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7b87b439 2022-04-26T06:43:24.827Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dbd659ef-7499-47c2-ac32-417d92047f82 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:43:24.827Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dbd659ef-7499-47c2-ac32-417d92047f82 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:43:24.831Z [qtp2085002312-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : etsi sol001 v2.5.1.tosca.policies.nfv.vduscalingaspectdeltas.1.0.policytype to dao for create 2022-04-26T06:43:24.925Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.PolicyTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=PolicyType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:25.020Z [qtp2085002312-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : etsi sol001 v2.5.1.tosca.policies.nfv.virtuallinkbitratescalingaspectdeltas.1.0.policytype to dao for create 2022-04-26T06:43:25.032Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.PolicyTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=PolicyType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:25.119Z [qtp2085002312-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : etsi sol001 v2.5.1.tosca.policies.nfv.vduinitialdelta.1.0.policytype to dao for create 2022-04-26T06:43:25.125Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.PolicyTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=PolicyType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:25.220Z [qtp2085002312-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : etsi sol001 v2.5.1.tosca.policies.nfv.virtuallinkbitrateinitialdelta.1.0.policytype to dao for create 2022-04-26T06:43:25.227Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.PolicyTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=PolicyType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:27.721Z [qtp2085002312-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : etsi sol001 v2.5.1.tosca.policies.nfv.affinityrule.1.0.policytype to dao for create 2022-04-26T06:43:27.735Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.PolicyTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=PolicyType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:27.856Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=dbd659ef-7499-47c2-ac32-417d92047f82 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:43:27.857Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=dbd659ef-7499-47c2-ac32-417d92047f82 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:43:28.532Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db5043fc-a3bf-4d38-9807-01f6f4522c46 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:43:28.533Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db5043fc-a3bf-4d38-9807-01f6f4522c46 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3a125fe2 2022-04-26T06:43:28.533Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db5043fc-a3bf-4d38-9807-01f6f4522c46 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:43:28.533Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db5043fc-a3bf-4d38-9807-01f6f4522c46 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:43:29.129Z [qtp2085002312-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : etsi sol001 v2.5.1.tosca.policies.nfv.antiaffinityrule.1.0.policytype to dao for create 2022-04-26T06:43:29.230Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.PolicyTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=PolicyType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:30.329Z [qtp2085002312-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : etsi sol001 v2.5.1.tosca.policies.nfv.securitygrouprule.1.0.policytype to dao for create 2022-04-26T06:43:30.628Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.PolicyTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=PolicyType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:30.931Z [qtp2085002312-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send POLICY_TYPE : etsi sol001 v2.5.1.tosca.policies.nfv.supportedvnfinterface.1.0.policytype to dao for create 2022-04-26T06:43:30.942Z [qtp2085002312-14] INFO o.o.s.b.m.o.impl.PolicyTypeOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Connecting model GraphNode [label=Model, parent: GraphElement [elementType=Node, action=NoAction]] to type GraphNode [label=PolicyType, parent: GraphElement [elementType=Node, action=NoAction]] 2022-04-26T06:43:30.949Z [qtp2085002312-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb all PolicyTypeDefinition(super=PolicyTypeDataDefinition [uniqueId=ETSI SOL001 v2.5.1.tosca.policies.nfv.SupportedVnfInterface.1.0.policytype, type=tosca.policies.nfv.SupportedVnfInterface, name=NFV Supported VnfInterface, icon=policy-nfv-supported-vnfinterface, version=1.0, targets=[tosca.nodes.nfv.VnfExtCp, tosca.nodes.nfv.VduCp], metadata=null, description=this policy type represents interfaces produced by a VNF, the details to access them and the applicable connection points to use to access these interfaces, creationTime=1650955410934, modificationTime=1650955410934], properties=[PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=ETSI SOL001 v2.5.1.tosca.policies.nfv.SupportedVnfInterface.1.0.policytype.interface_name, type=string, required=true, defaultValue=null, description=Identifies an interface produced by the VNF., schema=null, password=false, name=interface_name, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, model=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=[{"validValues":["vnf_indicator","vnf_configuration"]}], metadata=null), name=interface_name, constraints=[org.openecomp.sdc.be.model.tosca.constraints.ValidValuesConstraint@2a9d3d02]]], PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=ETSI SOL001 v2.5.1.tosca.policies.nfv.SupportedVnfInterface.1.0.policytype.details, type=tosca.datatypes.nfv.InterfaceDetails, required=false, defaultValue=null, description=Provide additional data to access the interface endpoint, schema=null, password=false, name=details, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, model=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=null, metadata=null), name=details, constraints=null]]], model=ETSI SOL001 v2.5.1) were created successfully!!! 2022-04-26T06:43:31.571Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=db5043fc-a3bf-4d38-9807-01f6f4522c46 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:43:31.571Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=db5043fc-a3bf-4d38-9807-01f6f4522c46 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:43:32.529Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf18f2dd-679a-49ca-896a-975df9543de6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:43:32.529Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf18f2dd-679a-49ca-896a-975df9543de6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@56655e88 2022-04-26T06:43:32.529Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf18f2dd-679a-49ca-896a-975df9543de6 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:43:32.529Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf18f2dd-679a-49ca-896a-975df9543de6 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:43:34.445Z [qtp2085002312-14] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=10417 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2022-04-26T06:43:24.028Z InvocationID=c7ad0ed0-2986-4590-a61c-aec17a06fba6 LogTimestamp=2022-04-26T06:43:34.445140Z ResponseDesc=Created Exiting. 2022-04-26T06:43:34.445Z [qtp2085002312-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/policytypes HTTP/1.1 SC="201" 2022-04-26T06:43:34.457Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=8f241cff-6bee-4c5b-8c80-f051d71754a6 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:43:34.456959Z InvocationID=1550f03b-5faa-4429-aa62-ef31b5edf41b LogTimestamp=2022-04-26T06:43:34.457021Z Entering 2022-04-26T06:43:34.457Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=8f241cff-6bee-4c5b-8c80-f051d71754a6 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.233.65.41 ServiceInstanceID=null EntryTimestamp=2022-04-26T06:43:34.456959Z InvocationID=1550f03b-5faa-4429-aa62-ef31b5edf41b LogTimestamp=2022-04-26T06:43:34.457021Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2022-04-26T06:43:34.672Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.Root.datatype to dao for create 2022-04-26T06:43:34.731Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : integer.datatype to dao for create 2022-04-26T06:43:34.734Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : string.datatype to dao for create 2022-04-26T06:43:34.736Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : boolean.datatype to dao for create 2022-04-26T06:43:34.741Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : float.datatype to dao for create 2022-04-26T06:43:34.744Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : list.datatype to dao for create 2022-04-26T06:43:34.746Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : map.datatype to dao for create 2022-04-26T06:43:34.829Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : json.datatype to dao for create 2022-04-26T06:43:34.832Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : scalar-unit.datatype to dao for create 2022-04-26T06:43:34.851Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : scalar-unit.size.datatype to dao for create 2022-04-26T06:43:34.861Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : scalar-unit.time.datatype to dao for create 2022-04-26T06:43:34.929Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : scalar-unit.frequency.datatype to dao for create 2022-04-26T06:43:34.934Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.Credential.datatype to dao for create 2022-04-26T06:43:34.941Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.PortMirroringConnectionPointDescription.datatype to dao for create 2022-04-26T06:43:34.945Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.TimeInterval.datatype to dao for create 2022-04-26T06:43:34.948Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.network.NetworkInfo.datatype to dao for create 2022-04-26T06:43:34.952Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.network.PortInfo.datatype to dao for create 2022-04-26T06:43:35.032Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.network.PortDef.datatype to dao for create 2022-04-26T06:43:35.034Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.network.PortSpec.datatype to dao for create 2022-04-26T06:43:35.039Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.network.AddressPair.datatype to dao for create 2022-04-26T06:43:35.042Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.network.subnet.HostRoute.datatype to dao for create 2022-04-26T06:43:35.130Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.network.AllocationPool.datatype to dao for create 2022-04-26T06:43:35.135Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.network.neutron.Subnet.datatype to dao for create 2022-04-26T06:43:35.149Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.novaServer.network.PortExtraProperties.datatype to dao for create 2022-04-26T06:43:35.229Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.novaServer.network.AddressInfo.datatype to dao for create 2022-04-26T06:43:35.233Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.neutron.port.FixedIps.datatype to dao for create 2022-04-26T06:43:35.238Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.FileInfo.datatype to dao for create 2022-04-26T06:43:35.243Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrail.network.rule.PortPairs.datatype to dao for create 2022-04-26T06:43:35.246Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrail.network.rule.Rule.datatype to dao for create 2022-04-26T06:43:35.249Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrail.network.rule.RuleList.datatype to dao for create 2022-04-26T06:43:35.252Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrail.network.rule.VirtualNetwork.datatype to dao for create 2022-04-26T06:43:35.330Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.network.neutron.SecurityRules.Rule.datatype to dao for create 2022-04-26T06:43:35.338Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.substitution.SubstitutionFiltering.datatype to dao for create 2022-04-26T06:43:35.345Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.network.rule.RefDataSequence.datatype to dao for create 2022-04-26T06:43:35.348Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.network.rule.RefData.datatype to dao for create 2022-04-26T06:43:35.350Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.network.rule.ref.data.IpamSubnet.datatype to dao for create 2022-04-26T06:43:35.353Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.network.rule.ref.data.IpamSubnetList.datatype to dao for create 2022-04-26T06:43:35.355Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.network.rule.IpamRefData.datatype to dao for create 2022-04-26T06:43:35.357Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.network.rule.SrcVirtualNetwork.datatype to dao for create 2022-04-26T06:43:35.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:43:35.374Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:43:35.429Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.network.rule.DstVirtualNetwork.datatype to dao for create 2022-04-26T06:43:35.435Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.network.rule.DstPortPairs.datatype to dao for create 2022-04-26T06:43:35.439Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.network.rule.SrcPortPairs.datatype to dao for create 2022-04-26T06:43:35.442Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.network.rule.ActionList.datatype to dao for create 2022-04-26T06:43:35.445Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.network.rule.Rule.datatype to dao for create 2022-04-26T06:43:35.450Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.network.rule.RuleList.datatype to dao for create 2022-04-26T06:43:35.528Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.network.contrail.port.StaticRoute.datatype to dao for create 2022-04-26T06:43:35.531Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.network.contrail.AddressPair.datatype to dao for create 2022-04-26T06:43:35.535Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.network.contrail.InterfaceData.datatype to dao for create 2022-04-26T06:43:35.538Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.machine.interface.Properties.datatype to dao for create 2022-04-26T06:43:35.539Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.Root.datatype to dao for create 2022-04-26T06:43:35.542Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.AssignmentRequirements.datatype to dao for create 2022-04-26T06:43:35.546Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.network.SubnetAssignments.datatype to dao for create 2022-04-26T06:43:35.554Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.network.IPv4SubnetAssignments.datatype to dao for create 2022-04-26T06:43:35.559Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=bf18f2dd-679a-49ca-896a-975df9543de6 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:43:35.559Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=bf18f2dd-679a-49ca-896a-975df9543de6 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:43:35.645Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.network.IPv6SubnetAssignments.datatype to dao for create 2022-04-26T06:43:35.649Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.network.NetworkAssignments.datatype to dao for create 2022-04-26T06:43:35.653Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.network.ProviderNetwork.datatype to dao for create 2022-04-26T06:43:35.658Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.network.NetworkFlows.datatype to dao for create 2022-04-26T06:43:35.660Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.network.VlanRequirements.datatype to dao for create 2022-04-26T06:43:35.733Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.network.IpRequirements.datatype to dao for create 2022-04-26T06:43:35.832Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.network.MacAssignments.datatype to dao for create 2022-04-26T06:43:35.835Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.CSProperties.datatype to dao for create 2022-04-26T06:43:35.842Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.NSCapabilities.datatype to dao for create 2022-04-26T06:43:35.850Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.NSSCapabilities.datatype to dao for create 2022-04-26T06:43:35.942Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.ServiceProfile.datatype to dao for create 2022-04-26T06:43:36.045Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.SliceProfile.datatype to dao for create 2022-04-26T06:43:36.145Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.EcompHoming.datatype to dao for create 2022-04-26T06:43:36.147Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.EcompNaming.datatype to dao for create 2022-04-26T06:43:36.149Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.network.MacRequirements.datatype to dao for create 2022-04-26T06:43:36.152Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.machine.subInterface.AddressPairIp.datatype to dao for create 2022-04-26T06:43:36.155Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.machine.subInterface.MacAddress.datatype to dao for create 2022-04-26T06:43:36.156Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.machine.subInterface.Properties.datatype to dao for create 2022-04-26T06:43:36.156Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.machine.subInterface.AddressPair.datatype to dao for create 2022-04-26T06:43:36.230Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03555dc7-094f-4d99-8b6b-10eaa269e30a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:43:36.231Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03555dc7-094f-4d99-8b6b-10eaa269e30a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3fe070f6 2022-04-26T06:43:36.231Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03555dc7-094f-4d99-8b6b-10eaa269e30a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:43:36.231Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03555dc7-094f-4d99-8b6b-10eaa269e30a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:43:36.232Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.machine.subInterface.AddressPairs.datatype to dao for create 2022-04-26T06:43:36.233Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.Naming.datatype to dao for create 2022-04-26T06:43:36.238Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.flavors.ComputeFlavor.datatype to dao for create 2022-04-26T06:43:36.244Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.flavors.LicenseFlavor.datatype to dao for create 2022-04-26T06:43:36.335Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.flavors.VendorInfo.datatype to dao for create 2022-04-26T06:43:36.346Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.flavors.DeploymentFlavor.datatype to dao for create 2022-04-26T06:43:36.349Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.ImageInfo.datatype to dao for create 2022-04-26T06:43:36.434Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : org.openecomp.datatypes.network.RelatedNetworksAssignments.datatype to dao for create 2022-04-26T06:43:36.443Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfcAdditionalConfigurableProperties.datatype to dao for create 2022-04-26T06:43:36.532Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfcConfigurableProperties.datatype to dao for create 2022-04-26T06:43:36.539Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.RequestedAdditionalCapability.datatype to dao for create 2022-04-26T06:43:36.548Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VirtualMemory.datatype to dao for create 2022-04-26T06:43:36.632Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VirtualCpuPinning.datatype to dao for create 2022-04-26T06:43:36.638Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VirtualCpu.datatype to dao for create 2022-04-26T06:43:36.642Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.LogicalNodeData.datatype to dao for create 2022-04-26T06:43:36.731Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VirtualNetworkInterfaceRequirements.datatype to dao for create 2022-04-26T06:43:36.737Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.injectFile.datatype to dao for create 2022-04-26T06:43:36.741Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.L2AddressData.datatype to dao for create 2022-04-26T06:43:36.743Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.L3AddressData.datatype to dao for create 2022-04-26T06:43:36.833Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.AddressData.datatype to dao for create 2022-04-26T06:43:36.837Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.ConnectivityType.datatype to dao for create 2022-04-26T06:43:36.840Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VlFlavour.datatype to dao for create 2022-04-26T06:43:36.843Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.ext.AddressPairs.datatype to dao for create 2022-04-26T06:43:36.845Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.ext.FloatingIP.datatype to dao for create 2022-04-26T06:43:36.846Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.ext.LocationInfo.datatype to dao for create 2022-04-26T06:43:36.847Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.ext.HostRouteInfo.datatype to dao for create 2022-04-26T06:43:36.931Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.ext.InjectData.datatype to dao for create 2022-04-26T06:43:36.937Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.ext.zte.WatchDog.datatype to dao for create 2022-04-26T06:43:36.938Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.ext.LocalAffinityOrAntiAffinityRule.datatype to dao for create 2022-04-26T06:43:36.939Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VduProfile.datatype to dao for create 2022-04-26T06:43:36.948Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.LinkBitrateRequirements.datatype to dao for create 2022-04-26T06:43:36.954Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.Qos.datatype to dao for create 2022-04-26T06:43:36.959Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.IpAllocationPool.datatype to dao for create 2022-04-26T06:43:36.962Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.L2ProtocolData.datatype to dao for create 2022-04-26T06:43:37.035Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.L3ProtocolData.datatype to dao for create 2022-04-26T06:43:37.042Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VirtualLinkProtocolData.datatype to dao for create 2022-04-26T06:43:37.046Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VlProfile.datatype to dao for create 2022-04-26T06:43:37.133Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VduLevel.datatype to dao for create 2022-04-26T06:43:37.140Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.ScaleInfo.datatype to dao for create 2022-04-26T06:43:37.230Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.ScalingAspect.datatype to dao for create 2022-04-26T06:43:37.234Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.InstantiationLevel.datatype to dao for create 2022-04-26T06:43:37.238Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfInstantiateOperationConfiguration.datatype to dao for create 2022-04-26T06:43:37.241Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfScaleOperationConfiguration.datatype to dao for create 2022-04-26T06:43:37.243Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfScaleToLevelOperationConfiguration.datatype to dao for create 2022-04-26T06:43:37.246Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfHealOperationConfiguration.datatype to dao for create 2022-04-26T06:43:37.248Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfTerminateOperationConfiguration.datatype to dao for create 2022-04-26T06:43:37.330Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfOperateOperationConfiguration.datatype to dao for create 2022-04-26T06:43:37.333Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfChangeFlavourOperationConfiguration.datatype to dao for create 2022-04-26T06:43:37.339Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfProfile.datatype to dao for create 2022-04-26T06:43:37.346Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.UriAuthority.datatype to dao for create 2022-04-26T06:43:37.429Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.UriComponents.datatype to dao for create 2022-04-26T06:43:37.432Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.InterfaceDetails.datatype to dao for create 2022-04-26T06:43:37.438Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VirtualLinkMonitoringParameter.datatype to dao for create 2022-04-26T06:43:37.443Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfcMonitoringParameter.datatype to dao for create 2022-04-26T06:43:37.448Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfChangeExtConnectivityOperationConfiguration.datatype to dao for create 2022-04-26T06:43:37.451Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send DATA_TYPE : tosca.datatypes.nfv.VnfMonitoringParameter.datatype to dao for create 2022-04-26T06:43:37.533Z [qtp2085002312-20] WARN o.o.s.b.c.impl.DataTypeImportManager RequestID=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Creating data type tosca.datatypes.nfv.VnfChangeCurrentPackageOperationConfiguration which derived from abstract data type with no properties 2022-04-26T06:43:37.534Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfChangeCurrentPackageOperationConfiguration.datatype to dao for create 2022-04-26T06:43:37.536Z [qtp2085002312-20] WARN o.o.s.b.c.impl.DataTypeImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 Creating data type tosca.datatypes.nfv.VnfCreateSnapshotOperationConfiguration which derived from abstract data type with no properties 2022-04-26T06:43:37.536Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfCreateSnapshotOperationConfiguration.datatype to dao for create 2022-04-26T06:43:37.539Z [qtp2085002312-20] WARN o.o.s.b.c.impl.DataTypeImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 Creating data type tosca.datatypes.nfv.VnfRevertToSnapshotOperationConfiguration which derived from abstract data type with no properties 2022-04-26T06:43:37.540Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfRevertToSnapshotOperationConfiguration.datatype to dao for create 2022-04-26T06:43:37.543Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfLcmOperationsConfiguration.datatype to dao for create 2022-04-26T06:43:37.547Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.CpProtocolData.datatype to dao for create 2022-04-26T06:43:37.631Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfAdditionalConfigurableProperties.datatype to dao for create 2022-04-26T06:43:37.635Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfmInterfaceInfo.datatype to dao for create 2022-04-26T06:43:37.639Z [qtp2085002312-20] WARN o.o.s.b.c.impl.DataTypeImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 Creating data type tosca.datatypes.nfv.OauthServerInfo which derived from abstract data type with no properties 2022-04-26T06:43:37.641Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.OauthServerInfo.datatype to dao for create 2022-04-26T06:43:37.645Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfConfigurableProperties.datatype to dao for create 2022-04-26T06:43:37.648Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfInfoModifiableAttributesExtensions.datatype to dao for create 2022-04-26T06:43:37.730Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfInfoModifiableAttributesMetadata.datatype to dao for create 2022-04-26T06:43:37.740Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfInfoModifiableAttributes.datatype to dao for create 2022-04-26T06:43:37.830Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VirtualLinkBitrateLevel.datatype to dao for create 2022-04-26T06:43:37.836Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfOperationAdditionalParameters.datatype to dao for create 2022-04-26T06:43:37.840Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VirtualFileStorageData.datatype to dao for create 2022-04-26T06:43:37.843Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VirtualObjectStorageData.datatype to dao for create 2022-04-26T06:43:37.845Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VirtualBlockStorageData.datatype to dao for create 2022-04-26T06:43:37.850Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.ChecksumData.datatype to dao for create 2022-04-26T06:43:37.854Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.SwImageData.datatype to dao for create 2022-04-26T06:43:37.864Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.NsVirtualLinkQos.datatype to dao for create 2022-04-26T06:43:37.867Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.ServiceAvailability.datatype to dao for create 2022-04-26T06:43:37.933Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.NsVlProfile.datatype to dao for create 2022-04-26T06:43:37.935Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.NsProfile.datatype to dao for create 2022-04-26T06:43:37.937Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.MciopProfile.datatype to dao for create 2022-04-26T06:43:37.942Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.BootDataVimSpecificProperties.datatype to dao for create 2022-04-26T06:43:37.945Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.KvpData.datatype to dao for create 2022-04-26T06:43:38.029Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.ContentOrFileData.datatype to dao for create 2022-04-26T06:43:38.031Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.BootData.datatype to dao for create 2022-04-26T06:43:38.034Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.ServicePortData.datatype to dao for create 2022-04-26T06:43:38.039Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.AdditionalServiceData.datatype to dao for create 2022-04-26T06:43:38.044Z [qtp2085002312-20] WARN o.o.s.b.c.impl.DataTypeImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 Creating data type tosca.datatypes.nfv.NsOperationAdditionalParameters which derived from abstract data type with no properties 2022-04-26T06:43:38.044Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.NsOperationAdditionalParameters.datatype to dao for create 2022-04-26T06:43:38.046Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.Mask.datatype to dao for create 2022-04-26T06:43:38.049Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.NsMonitoringParameter.datatype to dao for create 2022-04-26T06:43:38.130Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : onap.datatypes.partner.access_details.datatype to dao for create 2022-04-26T06:43:38.138Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : onap.datatypes.partner.bandwidth.datatype to dao for create 2022-04-26T06:43:38.147Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : onap.datatypes.partner.service_details.datatype to dao for create 2022-04-26T06:43:38.152Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : onap.datatypes.partner.sppartner_details.datatype to dao for create 2022-04-26T06:43:38.155Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 send DATA_TYPE : onap.datatypes.ToscaConceptIdentifier.datatype to dao for create 2022-04-26T06:43:38.157Z [qtp2085002312-20] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=500 all DataTypeDefinition(derivedFrom=null, constraints=null, properties=[PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=null, type=string, required=true, defaultValue=null, description=null, schema=null, password=false, name=name, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, model=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=null, metadata=null), name=name, constraints=null]], PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=null, type=string, required=true, defaultValue=null, description=null, schema=null, password=false, name=version, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, model=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=null, metadata=null), name=version, constraints=null]]]) were created successfully!!! 2022-04-26T06:43:38.178Z [qtp2085002312-20] WARN o.o.s.e.AbstractSdncException RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=900 Received less parameters than expected for error with messageId SVC4587, expected: 1, actual: 0. Missing parameters are padded with null values. 2022-04-26T06:43:38.179Z [qtp2085002312-20] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=409 RequestID=null ElapsedTime=3721 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=ERROR TargetEntity= ErrorCategory=WARN ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Conflict EntryTimestamp=2022-04-26T06:43:34.457Z InvocationID=b0456591-752d-4642-9ddd-a948565b3da9 ErrorCode=409 LogTimestamp=2022-04-26T06:43:38.178852Z ResponseDesc=Conflict Exiting. 2022-04-26T06:43:38.179Z [qtp2085002312-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="409" 2022-04-26T06:43:38.183Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=03555dc7-094f-4d99-8b6b-10eaa269e30a ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:43:38.184Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=03555dc7-094f-4d99-8b6b-10eaa269e30a ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:43:38.197Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=53a4c85e-c2c3-44bd-b2e2-a04b26c0010c InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/uploadType/capability ServiceInstanceID=null EntryTimestamp=2022-04-26T06:43:38.196882Z InvocationID=f0492ed2-796f-4067-9c93-72e3b91b6b95 LogTimestamp=2022-04-26T06:43:38.196942Z Entering 2022-04-26T06:43:38.197Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=53a4c85e-c2c3-44bd-b2e2-a04b26c0010c InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/uploadType/capability ServiceInstanceID=null EntryTimestamp=2022-04-26T06:43:38.196882Z InvocationID=f0492ed2-796f-4067-9c93-72e3b91b6b95 LogTimestamp=2022-04-26T06:43:38.196942Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/catalog/uploadType/capability HTTP/1.1 2022-04-26T06:43:38.216Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.Root to dao for create 2022-04-26T06:43:38.252Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.Attachment to dao for create 2022-04-26T06:43:38.344Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.Node to dao for create 2022-04-26T06:43:38.440Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.Container to dao for create 2022-04-26T06:43:38.742Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : org.openecomp.capabilities.PortMirroring to dao for create 2022-04-26T06:43:38.843Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.Endpoint to dao for create 2022-04-26T06:43:39.031Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5bfabba-ab61-4ff4-89b7-248a44d37a82 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:43:39.032Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5bfabba-ab61-4ff4-89b7-248a44d37a82 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7b8625d2 2022-04-26T06:43:39.032Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5bfabba-ab61-4ff4-89b7-248a44d37a82 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:43:39.032Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5bfabba-ab61-4ff4-89b7-248a44d37a82 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:43:39.135Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.DatabaseEndpoint to dao for create 2022-04-26T06:43:39.333Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.Endpoint.Public to dao for create 2022-04-26T06:43:39.439Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.Endpoint.Admin to dao for create 2022-04-26T06:43:39.543Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.Endpoint.Database to dao for create 2022-04-26T06:43:39.548Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.OperatingSystem to dao for create 2022-04-26T06:43:39.632Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.Scalable to dao for create 2022-04-26T06:43:39.644Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.network.Bindable to dao for create 2022-04-26T06:43:39.649Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.Container.Docker to dao for create 2022-04-26T06:43:39.835Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.network.Linkable to dao for create 2022-04-26T06:43:39.847Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : org.openecomp.capabilities.AllottedResource to dao for create 2022-04-26T06:43:39.933Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.nfv.Metric to dao for create 2022-04-26T06:43:39.940Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : org.openecomp.capabilities.Metric to dao for create 2022-04-26T06:43:40.135Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : org.openecomp.capabilities.metric.Ceilometer to dao for create 2022-04-26T06:43:40.157Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : org.openecomp.capabilities.metric.SnmpPolling to dao for create 2022-04-26T06:43:40.254Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : org.openecomp.capabilities.metric.SnmpTrap to dao for create 2022-04-26T06:43:40.354Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : org.openecomp.capabilities.Forwarder to dao for create 2022-04-26T06:43:40.374Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : org.openecomp.capabilities.VLANAssignment to dao for create 2022-04-26T06:43:40.431Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : org.openecomp.capabilities.RoutingConfiguration to dao for create 2022-04-26T06:43:40.439Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : org.openecomp.capabilities.FabricConfiguration to dao for create 2022-04-26T06:43:40.446Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.nfv.VirtualStorage to dao for create 2022-04-26T06:43:40.453Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.nfv.VirtualCompute to dao for create 2022-04-26T06:43:40.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58be9259-4b52-4d58-a27d-21f37e565f2c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:43:40.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58be9259-4b52-4d58-a27d-21f37e565f2c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@429f32b5 2022-04-26T06:43:40.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58be9259-4b52-4d58-a27d-21f37e565f2c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:43:40.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58be9259-4b52-4d58-a27d-21f37e565f2c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:43:40.468Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.nfv.VirtualBindable to dao for create 2022-04-26T06:43:40.474Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.nfv.VirtualLinkable to dao for create 2022-04-26T06:43:40.481Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.nfv.ExtVirtualLinkable to dao for create 2022-04-26T06:43:40.531Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.nfv.Forwarder to dao for create 2022-04-26T06:43:40.537Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.nfv.ext.GuestOs to dao for create 2022-04-26T06:43:40.543Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.nfv.ext.ImageFile to dao for create 2022-04-26T06:43:40.549Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.nfv.ext.LocalAttachment to dao for create 2022-04-26T06:43:40.556Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.nfv.ext.Compute.Container.Architecture to dao for create 2022-04-26T06:43:40.568Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send CAPABILITY_TYPE : tosca.capabilities.nfv.Forwarding to dao for create 2022-04-26T06:43:40.577Z [qtp2085002312-13] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb all CapabilityTypeDefinition(super=CapabilityTypeDataDefinition [uniqueId=null, description=null, type=tosca.capabilities.nfv.Forwarding, validSourceTypes=null, version=null, creationTime=null, modificationTime=null], derivedFrom=tosca.capabilities.Root, properties=null, model=null) were created successfully!!! 2022-04-26T06:43:40.583Z [qtp2085002312-13] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2386 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2022-04-26T06:43:38.197Z InvocationID=f688ca45-ddb9-4bf6-9dcd-622381419ede LogTimestamp=2022-04-26T06:43:40.583101Z ResponseDesc=Created Exiting. 2022-04-26T06:43:40.583Z [qtp2085002312-13] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/capability HTTP/1.1 SC="201" 2022-04-26T06:43:40.592Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=46c422fd-3f45-49bc-ac5b-6148cb13d0ef InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/uploadType/relationship ServiceInstanceID=null EntryTimestamp=2022-04-26T06:43:40.591626Z InvocationID=da3c4f9a-bb98-48cb-890d-92414046f8fa LogTimestamp=2022-04-26T06:43:40.592099Z Entering 2022-04-26T06:43:40.592Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=46c422fd-3f45-49bc-ac5b-6148cb13d0ef InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/uploadType/relationship ServiceInstanceID=null EntryTimestamp=2022-04-26T06:43:40.591626Z InvocationID=da3c4f9a-bb98-48cb-890d-92414046f8fa LogTimestamp=2022-04-26T06:43:40.592099Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/catalog/uploadType/relationship HTTP/1.1 2022-04-26T06:43:40.599Z [qtp2085002312-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.Root to dao for create 2022-04-26T06:43:40.876Z [qtp2085002312-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.ConnectsTo to dao for create 2022-04-26T06:43:41.454Z [qtp2085002312-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.RoutesTo to dao for create 2022-04-26T06:43:41.986Z [qtp2085002312-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.AttachesTo to dao for create 2022-04-26T06:43:42.285Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b1adcf0-e157-46d1-aace-c046f236b477 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:43:42.285Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b1adcf0-e157-46d1-aace-c046f236b477 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7ec48dd0 2022-04-26T06:43:42.285Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b1adcf0-e157-46d1-aace-c046f236b477 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:43:42.285Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b1adcf0-e157-46d1-aace-c046f236b477 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:43:42.601Z [qtp2085002312-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.HostedOn to dao for create 2022-04-26T06:43:44.623Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:43:45.291Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3b1adcf0-e157-46d1-aace-c046f236b477 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:43:45.291Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3b1adcf0-e157-46d1-aace-c046f236b477 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:43:45.292Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=3b1adcf0-e157-46d1-aace-c046f236b477 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:43:45.292Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=3b1adcf0-e157-46d1-aace-c046f236b477 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:43:45.436Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44fa870d-444b-43dc-a92c-9b1ecd47807c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:43:45.436Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44fa870d-444b-43dc-a92c-9b1ecd47807c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2420dec3 2022-04-26T06:43:45.437Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44fa870d-444b-43dc-a92c-9b1ecd47807c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:43:45.437Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44fa870d-444b-43dc-a92c-9b1ecd47807c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:43:46.185Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=44fa870d-444b-43dc-a92c-9b1ecd47807c ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:43:46.185Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=44fa870d-444b-43dc-a92c-9b1ecd47807c ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:43:46.333Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=384f64a6-b943-4e66-8cd6-ffe25d803986 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:43:46.334Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=384f64a6-b943-4e66-8cd6-ffe25d803986 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@55f35ad7 2022-04-26T06:43:46.334Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=384f64a6-b943-4e66-8cd6-ffe25d803986 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:43:46.334Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=384f64a6-b943-4e66-8cd6-ffe25d803986 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:43:46.655Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:43:48.685Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:43:49.340Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=384f64a6-b943-4e66-8cd6-ffe25d803986 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:43:49.340Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=384f64a6-b943-4e66-8cd6-ffe25d803986 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:43:49.341Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=384f64a6-b943-4e66-8cd6-ffe25d803986 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:43:49.341Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=384f64a6-b943-4e66-8cd6-ffe25d803986 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:43:49.828Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e8e8c7e-374d-4e71-abb8-9e0a3fb9ec8e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:43:49.828Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e8e8c7e-374d-4e71-abb8-9e0a3fb9ec8e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@f83f7b4 2022-04-26T06:43:49.829Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e8e8c7e-374d-4e71-abb8-9e0a3fb9ec8e PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:43:49.829Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e8e8c7e-374d-4e71-abb8-9e0a3fb9ec8e PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:43:50.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:43:50.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:43:50.739Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:43:50.787Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=9e8e8c7e-374d-4e71-abb8-9e0a3fb9ec8e ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:43:50.788Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=9e8e8c7e-374d-4e71-abb8-9e0a3fb9ec8e ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:43:51.722Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73e080c2-e694-4c40-90bc-a329312505ee PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:43:51.722Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73e080c2-e694-4c40-90bc-a329312505ee PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@55f4df08 2022-04-26T06:43:51.723Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73e080c2-e694-4c40-90bc-a329312505ee PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:43:51.723Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73e080c2-e694-4c40-90bc-a329312505ee PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:43:52.218Z [qtp2085002312-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.DependsOn to dao for create 2022-04-26T06:43:52.417Z [qtp2085002312-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.network.LinksTo to dao for create 2022-04-26T06:43:52.686Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:43:52.687Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:43:52.687Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:43:53.918Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6b2590e-440f-4f7a-a9b0-c79449f1f516 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:43:53.918Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6b2590e-440f-4f7a-a9b0-c79449f1f516 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1a1e0283 2022-04-26T06:43:53.919Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6b2590e-440f-4f7a-a9b0-c79449f1f516 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:43:53.919Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6b2590e-440f-4f7a-a9b0-c79449f1f516 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:43:54.527Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:43:55.921Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e37043b1-8f25-4c49-8195-f147b4ed7936 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:43:55.921Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e37043b1-8f25-4c49-8195-f147b4ed7936 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@ab06823 2022-04-26T06:43:55.921Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e37043b1-8f25-4c49-8195-f147b4ed7936 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:43:55.922Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e37043b1-8f25-4c49-8195-f147b4ed7936 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:43:56.582Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:43:58.720Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:43:59.017Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbe7d38a-9825-4ca7-9c7f-070126173b91 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:43:59.017Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbe7d38a-9825-4ca7-9c7f-070126173b91 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@186742ca 2022-04-26T06:43:59.017Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbe7d38a-9825-4ca7-9c7f-070126173b91 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:43:59.017Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbe7d38a-9825-4ca7-9c7f-070126173b91 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:44:00.328Z [qtp2085002312-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.network.BindsTo to dao for create 2022-04-26T06:44:00.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4136e390-980d-4fe2-95cf-ff4e39a71cac PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:44:00.632Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4136e390-980d-4fe2-95cf-ff4e39a71cac PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6b18c251 2022-04-26T06:44:00.632Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4136e390-980d-4fe2-95cf-ff4e39a71cac PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:44:00.632Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4136e390-980d-4fe2-95cf-ff4e39a71cac PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:44:00.916Z [qtp2085002312-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : org.openecomp.relationships.AttachesTo to dao for create 2022-04-26T06:44:02.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1e4f09e-0791-427a-959a-80557a2ee0df PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:44:02.429Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1e4f09e-0791-427a-959a-80557a2ee0df PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f8cfab 2022-04-26T06:44:02.430Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1e4f09e-0791-427a-959a-80557a2ee0df PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:44:02.430Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1e4f09e-0791-427a-959a-80557a2ee0df PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:44:03.018Z [qtp2085002312-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : org.openecomp.relationships.VolumeAttachesTo to dao for create 2022-04-26T06:44:04.536Z [qtp2085002312-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : org.openecomp.relationships.ForwardsTo to dao for create 2022-04-26T06:44:05.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:44:05.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:44:05.436Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f1e4f09e-0791-427a-959a-80557a2ee0df ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:44:05.436Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f1e4f09e-0791-427a-959a-80557a2ee0df ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:44:05.437Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=f1e4f09e-0791-427a-959a-80557a2ee0df ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:44:05.437Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=f1e4f09e-0791-427a-959a-80557a2ee0df ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:44:06.238Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=411aebed-407b-40dc-8b44-a06c2de24c78 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:44:06.239Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=411aebed-407b-40dc-8b44-a06c2de24c78 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8004ec7 2022-04-26T06:44:06.239Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=411aebed-407b-40dc-8b44-a06c2de24c78 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:44:06.240Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=411aebed-407b-40dc-8b44-a06c2de24c78 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:44:06.559Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:44:07.680Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:44:08.590Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:44:09.277Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=411aebed-407b-40dc-8b44-a06c2de24c78 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:44:09.278Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=411aebed-407b-40dc-8b44-a06c2de24c78 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:44:09.431Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ed1d44a-250b-4cb1-bd6f-931dbde184d5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:44:09.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ed1d44a-250b-4cb1-bd6f-931dbde184d5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f4795e 2022-04-26T06:44:09.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ed1d44a-250b-4cb1-bd6f-931dbde184d5 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:44:09.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ed1d44a-250b-4cb1-bd6f-931dbde184d5 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:44:10.623Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:44:12.468Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=2ed1d44a-250b-4cb1-bd6f-931dbde184d5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:44:12.468Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=2ed1d44a-250b-4cb1-bd6f-931dbde184d5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:44:12.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f75fa559-7347-494b-856e-2413c831e2ee PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:44:12.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f75fa559-7347-494b-856e-2413c831e2ee PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4aca733b 2022-04-26T06:44:12.639Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f75fa559-7347-494b-856e-2413c831e2ee PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:44:12.639Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f75fa559-7347-494b-856e-2413c831e2ee PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:44:12.680Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:44:14.195Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=f75fa559-7347-494b-856e-2413c831e2ee ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:44:14.195Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=f75fa559-7347-494b-856e-2413c831e2ee ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:44:14.354Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e613f31-86e9-4824-aa34-df94d443e0b9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:44:14.355Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e613f31-86e9-4824-aa34-df94d443e0b9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5d50a557 2022-04-26T06:44:14.355Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e613f31-86e9-4824-aa34-df94d443e0b9 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:44:14.355Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e613f31-86e9-4824-aa34-df94d443e0b9 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:44:14.786Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:44:16.275Z [qtp2085002312-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : org.openecomp.relationships.AssignsTo to dao for create 2022-04-26T06:44:16.440Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85f57e9b-2ae8-473d-90ed-c1704e195632 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:44:16.440Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85f57e9b-2ae8-473d-90ed-c1704e195632 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2ec95ffa 2022-04-26T06:44:16.441Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85f57e9b-2ae8-473d-90ed-c1704e195632 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:44:16.441Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85f57e9b-2ae8-473d-90ed-c1704e195632 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:44:18.300Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:44:19.447Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=85f57e9b-2ae8-473d-90ed-c1704e195632 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:44:19.447Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=85f57e9b-2ae8-473d-90ed-c1704e195632 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:44:19.448Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=85f57e9b-2ae8-473d-90ed-c1704e195632 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:44:19.448Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=85f57e9b-2ae8-473d-90ed-c1704e195632 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:44:20.119Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57a51bf9-49e3-4721-8020-78cc62150698 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:44:20.120Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57a51bf9-49e3-4721-8020-78cc62150698 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3783b2b3 2022-04-26T06:44:20.120Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57a51bf9-49e3-4721-8020-78cc62150698 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:44:20.120Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57a51bf9-49e3-4721-8020-78cc62150698 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:44:20.331Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:44:20.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:44:20.379Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:44:22.385Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:44:23.159Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=57a51bf9-49e3-4721-8020-78cc62150698 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:44:23.160Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=57a51bf9-49e3-4721-8020-78cc62150698 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:44:23.923Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b57c7bb3-5b7a-4bdf-989e-76dd54b581f3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:44:23.923Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b57c7bb3-5b7a-4bdf-989e-76dd54b581f3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@46af2234 2022-04-26T06:44:23.923Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b57c7bb3-5b7a-4bdf-989e-76dd54b581f3 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:44:23.923Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b57c7bb3-5b7a-4bdf-989e-76dd54b581f3 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:44:24.519Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:44:25.527Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=b57c7bb3-5b7a-4bdf-989e-76dd54b581f3 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:44:25.527Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=b57c7bb3-5b7a-4bdf-989e-76dd54b581f3 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:44:26.322Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a8d4f4e-a412-4552-b06c-420c4999e391 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:44:26.322Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a8d4f4e-a412-4552-b06c-420c4999e391 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8c1bb42 2022-04-26T06:44:26.322Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a8d4f4e-a412-4552-b06c-420c4999e391 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:44:26.323Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a8d4f4e-a412-4552-b06c-420c4999e391 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:44:26.724Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:44:29.134Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:44:29.328Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=7a8d4f4e-a412-4552-b06c-420c4999e391 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:44:29.328Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=7a8d4f4e-a412-4552-b06c-420c4999e391 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:44:29.328Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=7a8d4f4e-a412-4552-b06c-420c4999e391 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:44:29.328Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=7a8d4f4e-a412-4552-b06c-420c4999e391 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:44:30.132Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68f5c5ab-8237-4929-82a0-b417d3727c1b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:44:30.132Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68f5c5ab-8237-4929-82a0-b417d3727c1b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7c8dac4b 2022-04-26T06:44:30.132Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68f5c5ab-8237-4929-82a0-b417d3727c1b PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:44:30.132Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68f5c5ab-8237-4929-82a0-b417d3727c1b PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:44:31.632Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:44:31.726Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=68f5c5ab-8237-4929-82a0-b417d3727c1b ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:44:31.726Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=68f5c5ab-8237-4929-82a0-b417d3727c1b ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:44:32.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0fe3b5e-1356-41dc-8bbe-e39646e46dfa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:44:32.433Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0fe3b5e-1356-41dc-8bbe-e39646e46dfa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@da345aa 2022-04-26T06:44:32.433Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0fe3b5e-1356-41dc-8bbe-e39646e46dfa PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:44:32.433Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0fe3b5e-1356-41dc-8bbe-e39646e46dfa PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:44:34.452Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:44:35.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:44:35.383Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:44:35.534Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c53900bc-7bf0-46d0-bff5-25693341f95d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:44:35.534Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c53900bc-7bf0-46d0-bff5-25693341f95d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8bde4f7 2022-04-26T06:44:35.534Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c53900bc-7bf0-46d0-bff5-25693341f95d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:44:35.534Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c53900bc-7bf0-46d0-bff5-25693341f95d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:44:37.335Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:44:38.442Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bedac971-8e2b-4662-b576-be3f4a1e912a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:44:38.442Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bedac971-8e2b-4662-b576-be3f4a1e912a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@19c3ee5f 2022-04-26T06:44:38.442Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bedac971-8e2b-4662-b576-be3f4a1e912a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:44:38.443Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bedac971-8e2b-4662-b576-be3f4a1e912a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:44:40.942Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:44:41.447Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=bedac971-8e2b-4662-b576-be3f4a1e912a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:44:41.448Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=bedac971-8e2b-4662-b576-be3f4a1e912a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:44:41.448Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=bedac971-8e2b-4662-b576-be3f4a1e912a ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:44:41.448Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=bedac971-8e2b-4662-b576-be3f4a1e912a ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:44:41.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce63e242-d8ef-48f9-9268-3701eaa4a10b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:44:41.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce63e242-d8ef-48f9-9268-3701eaa4a10b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3e1375e4 2022-04-26T06:44:41.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce63e242-d8ef-48f9-9268-3701eaa4a10b PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:44:41.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce63e242-d8ef-48f9-9268-3701eaa4a10b PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:44:43.182Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=ce63e242-d8ef-48f9-9268-3701eaa4a10b ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:44:43.182Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=ce63e242-d8ef-48f9-9268-3701eaa4a10b ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:44:43.329Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f73a3f2-5cf8-47b2-a2a9-cf69bbf35943 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:44:43.329Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f73a3f2-5cf8-47b2-a2a9-cf69bbf35943 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4ef73a13 2022-04-26T06:44:43.329Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f73a3f2-5cf8-47b2-a2a9-cf69bbf35943 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:44:43.329Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f73a3f2-5cf8-47b2-a2a9-cf69bbf35943 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:44:44.546Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:44:46.335Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=9f73a3f2-5cf8-47b2-a2a9-cf69bbf35943 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:44:46.335Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=9f73a3f2-5cf8-47b2-a2a9-cf69bbf35943 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:44:46.336Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=9f73a3f2-5cf8-47b2-a2a9-cf69bbf35943 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:44:46.336Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=9f73a3f2-5cf8-47b2-a2a9-cf69bbf35943 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:44:46.476Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03021b15-a211-45a2-9e0b-777b90911233 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:44:46.476Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03021b15-a211-45a2-9e0b-777b90911233 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@48409484 2022-04-26T06:44:46.477Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03021b15-a211-45a2-9e0b-777b90911233 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:44:46.477Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03021b15-a211-45a2-9e0b-777b90911233 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:44:48.150Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:44:49.505Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=03021b15-a211-45a2-9e0b-777b90911233 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:44:49.506Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=03021b15-a211-45a2-9e0b-777b90911233 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:44:49.939Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=039081b4-3231-4afa-9d60-aa7a3c2f405f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:44:49.940Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=039081b4-3231-4afa-9d60-aa7a3c2f405f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@13618e2e 2022-04-26T06:44:49.940Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=039081b4-3231-4afa-9d60-aa7a3c2f405f PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:44:49.940Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=039081b4-3231-4afa-9d60-aa7a3c2f405f PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:44:50.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:44:50.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:44:50.591Z [cluster1-timeouter-0] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.233.72.7:9042 com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra.onap/10.233.72.7:9042] Operation timed out at com.datastax.driver.core.Connection$Future.onTimeout(Connection.java:1526) at com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1605) at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:669) at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:744) at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:469) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:44:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:44:52.686Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:44:52.686Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:44:52.972Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=039081b4-3231-4afa-9d60-aa7a3c2f405f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:44:52.973Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=039081b4-3231-4afa-9d60-aa7a3c2f405f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:44:53.417Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:44:53.815Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f398126-ab15-436a-8d8c-8288194e6653 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:44:53.815Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f398126-ab15-436a-8d8c-8288194e6653 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@28194555 2022-04-26T06:44:53.816Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f398126-ab15-436a-8d8c-8288194e6653 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:44:53.816Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f398126-ab15-436a-8d8c-8288194e6653 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:44:55.229Z [JanusGraph Cluster-timeouter-0] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.233.72.7:9042 com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra.onap/10.233.72.7:9042] Operation timed out at com.datastax.driver.core.Connection$Future.onTimeout(Connection.java:1526) at com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1605) at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:669) at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:744) at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:469) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:44:56.854Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3f398126-ab15-436a-8d8c-8288194e6653 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:44:56.855Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3f398126-ab15-436a-8d8c-8288194e6653 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:44:57.718Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=681bf452-e907-44b9-af07-639cd6f7b30d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:44:57.719Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=681bf452-e907-44b9-af07-639cd6f7b30d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@11c85bff 2022-04-26T06:44:57.719Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=681bf452-e907-44b9-af07-639cd6f7b30d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:44:57.719Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=681bf452-e907-44b9-af07-639cd6f7b30d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:44:58.622Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:45:00.748Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=681bf452-e907-44b9-af07-639cd6f7b30d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:45:00.749Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=681bf452-e907-44b9-af07-639cd6f7b30d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:45:01.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e629dce2-4010-4d96-bb63-f1f6ad440502 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:45:01.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e629dce2-4010-4d96-bb63-f1f6ad440502 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f0be7b7 2022-04-26T06:45:01.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e629dce2-4010-4d96-bb63-f1f6ad440502 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:45:01.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e629dce2-4010-4d96-bb63-f1f6ad440502 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:45:03.984Z [cluster2-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.233.72.7:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:45:04.081Z [JanusGraph Cluster-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:45:04.081Z [cluster2-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.233.72.7:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:45:04.082Z [JanusGraph Cluster-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 2000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:45:04.569Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e629dce2-4010-4d96-bb63-f1f6ad440502 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:45:04.570Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e629dce2-4010-4d96-bb63-f1f6ad440502 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:45:05.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:45:05.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:45:05.430Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc0d8f82-c00b-436f-be76-4a9a0dc9d4e2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:45:05.430Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc0d8f82-c00b-436f-be76-4a9a0dc9d4e2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f1c397a 2022-04-26T06:45:05.430Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc0d8f82-c00b-436f-be76-4a9a0dc9d4e2 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:45:05.430Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc0d8f82-c00b-436f-be76-4a9a0dc9d4e2 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:45:07.029Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:45:07.682Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:45:08.468Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=dc0d8f82-c00b-436f-be76-4a9a0dc9d4e2 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:45:08.468Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=dc0d8f82-c00b-436f-be76-4a9a0dc9d4e2 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:45:08.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0d4c70c-477b-400e-bcf4-9b22d2d27c4d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:45:08.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0d4c70c-477b-400e-bcf4-9b22d2d27c4d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1329bca8 2022-04-26T06:45:08.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0d4c70c-477b-400e-bcf4-9b22d2d27c4d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:45:08.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0d4c70c-477b-400e-bcf4-9b22d2d27c4d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:45:10.977Z [cluster2-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.233.72.7:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: java.lang.RuntimeException: org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:45:11.633Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f0d4c70c-477b-400e-bcf4-9b22d2d27c4d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:45:11.634Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f0d4c70c-477b-400e-bcf4-9b22d2d27c4d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:45:11.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7466dd5e-90da-419b-9e28-01cf9720d8cb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:45:11.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7466dd5e-90da-419b-9e28-01cf9720d8cb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@65956268 2022-04-26T06:45:11.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7466dd5e-90da-419b-9e28-01cf9720d8cb PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:45:11.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7466dd5e-90da-419b-9e28-01cf9720d8cb PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:45:13.475Z [JanusGraph Cluster-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2022-04-26T06:45:13.476Z [JanusGraph Cluster-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.233.72.7:9042, scheduling retry in 4000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.233.72.7:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:45:14.801Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=7466dd5e-90da-419b-9e28-01cf9720d8cb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:45:14.802Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=7466dd5e-90da-419b-9e28-01cf9720d8cb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:45:20.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:45:20.378Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:45:21.833Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:45:30.178Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e979586d-ef0e-4f16-864b-2bb5c1e0dec9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:45:30.178Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e979586d-ef0e-4f16-864b-2bb5c1e0dec9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@168ce621 2022-04-26T06:45:30.178Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e979586d-ef0e-4f16-864b-2bb5c1e0dec9 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:45:30.178Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e979586d-ef0e-4f16-864b-2bb5c1e0dec9 PartnerName=SDC.BE The cassandra down nodes number is 0 2022-04-26T06:45:31.036Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=e979586d-ef0e-4f16-864b-2bb5c1e0dec9 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:45:31.037Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=e979586d-ef0e-4f16-864b-2bb5c1e0dec9 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:45:35.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:45:35.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:45:48.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2bb5eb2-959f-467d-a436-ea11a8d72d88 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:45:48.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2bb5eb2-959f-467d-a436-ea11a8d72d88 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@419d4283 2022-04-26T06:45:48.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2bb5eb2-959f-467d-a436-ea11a8d72d88 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:45:48.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2bb5eb2-959f-467d-a436-ea11a8d72d88 PartnerName=SDC.BE The cassandra down nodes number is 0 2022-04-26T06:45:49.439Z [qtp2085002312-15] ERROR o.j.g.database.StandardJanusGraph RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Could not commit transaction [435] due to storage exception in commit org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT1M40S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 126 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 127 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:45:49.440Z [qtp2085002312-15] ERROR o.j.g.database.StandardJanusGraph RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Could not commit transaction [435] due to exception org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT1M40S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 126 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 127 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:45:49.440Z [qtp2085002312-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : org.openecomp.relationships.RoutesTo to dao for create 2022-04-26T06:45:50.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:45:50.374Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:45:51.679Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f2bb5eb2-959f-467d-a436-ea11a8d72d88 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:45:51.680Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f2bb5eb2-959f-467d-a436-ea11a8d72d88 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:45:51.680Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=f2bb5eb2-959f-467d-a436-ea11a8d72d88 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:45:51.680Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=f2bb5eb2-959f-467d-a436-ea11a8d72d88 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2022-04-26T06:45:52.220Z [qtp2085002312-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.nfv.VirtualBindsTo to dao for create 2022-04-26T06:45:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:45:52.686Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:45:52.686Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:45:58.078Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:45:59.583Z [qtp2085002312-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.nfv.VirtualLinksTo to dao for create 2022-04-26T06:46:00.783Z [qtp2085002312-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.nfv.AttachesTo to dao for create 2022-04-26T06:46:02.884Z [qtp2085002312-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.nfv.Vdu.AttachedTo to dao for create 2022-04-26T06:46:05.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:46:05.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:46:05.582Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 1 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 1 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 1 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:46:05.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d52c3168-73e7-46a2-93bb-2fc3952df6b7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:46:05.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d52c3168-73e7-46a2-93bb-2fc3952df6b7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f464029 2022-04-26T06:46:05.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d52c3168-73e7-46a2-93bb-2fc3952df6b7 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:46:05.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d52c3168-73e7-46a2-93bb-2fc3952df6b7 PartnerName=SDC.BE The cassandra down nodes number is 0 2022-04-26T06:46:07.639Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:46:07.672Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:46:08.812Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=d52c3168-73e7-46a2-93bb-2fc3952df6b7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:46:08.813Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=d52c3168-73e7-46a2-93bb-2fc3952df6b7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:46:09.729Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:46:11.979Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 1 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 1 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 1 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:46:12.284Z [qtp2085002312-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.nfv.ForwardTo to dao for create 2022-04-26T06:46:16.477Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteFailureException: Cassandra failure during write query at consistency ALL (3 responses were required but only 2 replica responded, 1 failed) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteFailureException: Cassandra failure during write query at consistency ALL (3 responses were required but only 2 replica responded, 1 failed) at com.datastax.driver.core.exceptions.WriteFailureException.copy(WriteFailureException.java:186) at com.datastax.driver.core.Responses$Error.asException(Responses.java:171) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteFailureException: Cassandra failure during write query at consistency ALL (3 responses were required but only 2 replica responded, 1 failed) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:112) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:46:18.533Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:46:19.182Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79bb19eb-2e82-474f-8492-1a0fc4cb6917 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:46:19.182Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79bb19eb-2e82-474f-8492-1a0fc4cb6917 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2e461e13 2022-04-26T06:46:19.183Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79bb19eb-2e82-474f-8492-1a0fc4cb6917 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:46:19.183Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79bb19eb-2e82-474f-8492-1a0fc4cb6917 PartnerName=SDC.BE The cassandra down nodes number is 0 2022-04-26T06:46:20.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:46:20.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:46:20.615Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:46:22.214Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=79bb19eb-2e82-474f-8492-1a0fc4cb6917 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:46:22.214Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=79bb19eb-2e82-474f-8492-1a0fc4cb6917 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:46:22.716Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:46:24.821Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:46:26.925Z [qtp2085002312-15] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.RelationshipTypeOperation.updateRelationshipType(RelationshipTypeOperation.java:300) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypesByDao$6(RelationshipTypeImportManager.java:99) at org.openecomp.sdc.be.components.impl.CommonImportManager.updateElementType(CommonImportManager.java:464) at org.openecomp.sdc.be.components.impl.CommonImportManager.handleType(CommonImportManager.java:337) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypesByDao(CommonImportManager.java:277) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypesByDao(RelationshipTypeImportManager.java:92) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.lambda$createRelationshipTypes$1(RelationshipTypeImportManager.java:67) at org.openecomp.sdc.be.components.impl.CommonImportManager.createElementTypes(CommonImportManager.java:534) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:66) at org.openecomp.sdc.be.components.impl.RelationshipTypeImportManager.createRelationshipTypes(RelationshipTypeImportManager.java:58) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$createRelationshipTypes$11(TypesUploadServlet.java:454) at org.openecomp.sdc.be.servlets.TypesUploadServlet.buildStatusForElementTypeCreate(TypesUploadServlet.java:411) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createRelationshipTypes(TypesUploadServlet.java:455) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:316) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType_aroundBody2(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure3.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadRelationshipType(TypesUploadServlet.java:160) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadRelationshipType() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:46:29.014Z [qtp2085002312-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send RELATIONSHIP_TYPE : tosca.relationships.nfv.ContainerBindsTo to dao for create 2022-04-26T06:46:31.676Z [qtp2085002312-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb all RelationshipTypeDefinition(super=RelationshipTypeDataDefinition [uniqueId=tosca.relationships.nfv.ContainerBindsTo, description=Represents an association relationship between Vdu.OsContainerGroup and VduCp node types, type=tosca.relationships.nfv.ContainerBindsTo, validSourceTypes=null, version=null, originUI=null, creationTime=1650954014090, modificationTime=1650955589331], derivedFrom=tosca.relationships.DependsOn, properties=null, model=null) were created successfully!!! 2022-04-26T06:46:31.680Z [qtp2085002312-15] WARN o.o.s.be.servlets.TypesUploadServlet RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb #uploadRelationshipType(/tmp/rep11887489190545678227tmp, 'Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/catalog/uploadType/relationship)@5cce147', 'jh0003', NULL, false): org.apache.cxf.jaxrs.impl.ResponseImpl@49e1d95a in 2.85min (too slow!) 2022-04-26T06:46:31.681Z [qtp2085002312-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=171088 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=COMPLETE ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2022-04-26T06:43:40.593Z InvocationID=71d731b7-7716-4711-81bc-81284af469c5 LogTimestamp=2022-04-26T06:46:31.681104Z ResponseDesc=Created Exiting. 2022-04-26T06:46:31.681Z [qtp2085002312-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/relationship HTTP/1.1 SC="201" 2022-04-26T06:46:31.989Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=24e1cc3d-13e2-4b32-98b0-58c4ddbfed16 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/uploadType/interfaceLifecycle ServiceInstanceID=null EntryTimestamp=2022-04-26T06:46:31.989281Z InvocationID=25c6bc13-e363-4622-9990-d5eafa9f94ce LogTimestamp=2022-04-26T06:46:31.989347Z Entering 2022-04-26T06:46:31.989Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=24e1cc3d-13e2-4b32-98b0-58c4ddbfed16 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/uploadType/interfaceLifecycle ServiceInstanceID=null EntryTimestamp=2022-04-26T06:46:31.989281Z InvocationID=25c6bc13-e363-4622-9990-d5eafa9f94ce LogTimestamp=2022-04-26T06:46:31.989347Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/catalog/uploadType/interfaceLifecycle HTTP/1.1 2022-04-26T06:46:31.996Z [qtp2085002312-18] INFO o.o.s.b.c.i.InterfaceLifecycleTypeImportManager RequestID=null ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2022-04-26T06:46:31.990Z InvocationID=759e87bd-e660-40f1-87a4-be5dab5accf3 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb send interfaceDefinition tosca.interfaces.Root to dao for create 2022-04-26T06:46:32.001Z [qtp2085002312-18] ERROR o.o.s.b.m.o.i.InterfaceLifecycleOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle TargetServiceName=null EntryTimestamp=2022-04-26T06:46:31.990Z InvocationID=759e87bd-e660-40f1-87a4-be5dab5accf3 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Going to execute rollback on graph. 2022-04-26T06:46:32.002Z [qtp2085002312-18] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle TargetServiceName=null EntryTimestamp=2022-04-26T06:46:31.990Z InvocationID=759e87bd-e660-40f1-87a4-be5dab5accf3 ErrorCode=500 Going to execute rollback on graph. 2022-04-26T06:46:32.002Z [qtp2085002312-18] INFO o.o.s.b.c.i.InterfaceLifecycleTypeImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle TargetServiceName=null EntryTimestamp=2022-04-26T06:46:31.990Z InvocationID=759e87bd-e660-40f1-87a4-be5dab5accf3 ErrorCode=500 failed to create interface : tosca.interfaces.Root error: ENTITY_ALREADY_EXISTS 2022-04-26T06:46:32.004Z [qtp2085002312-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=409 RequestID=null ElapsedTime=14 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb StatusCode=ERROR TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/interfaceLifecycle TargetServiceName=null ErrorDesc=Conflict EntryTimestamp=2022-04-26T06:46:31.990Z InvocationID=759e87bd-e660-40f1-87a4-be5dab5accf3 ErrorCode=409 LogTimestamp=2022-04-26T06:46:32.004287Z ResponseDesc=Conflict Exiting. 2022-04-26T06:46:32.004Z [qtp2085002312-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/interfaceLifecycle HTTP/1.1 SC="409" 2022-04-26T06:46:32.011Z [qtp2085002312-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=9cc53ab4-b697-43df-abff-0ea88f9ca108 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/uploadType/categories ServiceInstanceID=null EntryTimestamp=2022-04-26T06:46:32.011182Z InvocationID=85ccb31e-0059-4ee3-8992-7278ca5179da LogTimestamp=2022-04-26T06:46:32.011231Z Entering 2022-04-26T06:46:32.011Z [qtp2085002312-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=9cc53ab4-b697-43df-abff-0ea88f9ca108 InstanceID=eca66d38-83c5-4950-beb0-92680d972828 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb ClientIPAddress=10.233.66.217 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.233.66.217 ServerIPAddress=10.233.65.41 ServiceName=v1/catalog/uploadType/categories ServiceInstanceID=null EntryTimestamp=2022-04-26T06:46:32.011182Z InvocationID=85ccb31e-0059-4ee3-8992-7278ca5179da LogTimestamp=2022-04-26T06:46:32.011231Z localAddr=10.233.65.41 remoteAddr=10.233.66.217 POST /sdc2/rest/v1/catalog/uploadType/categories HTTP/1.1 2022-04-26T06:46:32.033Z [qtp2085002312-12] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null Going to execute rollback on graph. 2022-04-26T06:46:33.378Z [qtp2085002312-12] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Going to execute rollback on graph. 2022-04-26T06:46:34.190Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d921e7ed-8e7f-4d65-995d-69ae40451d4e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:46:34.191Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d921e7ed-8e7f-4d65-995d-69ae40451d4e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@486e879f 2022-04-26T06:46:34.191Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d921e7ed-8e7f-4d65-995d-69ae40451d4e PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:46:34.191Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d921e7ed-8e7f-4d65-995d-69ae40451d4e PartnerName=SDC.BE The cassandra down nodes number is 0 2022-04-26T06:46:35.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:46:35.382Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:46:36.189Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:293) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:269) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createSubCategoriesDao(CategoriesImportManager.java:161) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.isCreatedNewSubCategory(CategoriesImportManager.java:102) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:84) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:46:37.231Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=d921e7ed-8e7f-4d65-995d-69ae40451d4e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:46:37.233Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=d921e7ed-8e7f-4d65-995d-69ae40451d4e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:46:38.252Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:293) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:269) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createSubCategoriesDao(CategoriesImportManager.java:161) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.isCreatedNewSubCategory(CategoriesImportManager.java:102) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:84) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:46:39.886Z [qtp2085002312-12] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Going to execute rollback on graph. 2022-04-26T06:46:40.390Z [qtp2085002312-12] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Going to execute rollback on graph. 2022-04-26T06:46:40.780Z [qtp2085002312-12] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Going to execute rollback on graph. 2022-04-26T06:46:41.287Z [qtp2085002312-12] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Going to execute rollback on graph. 2022-04-26T06:46:41.889Z [qtp2085002312-12] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Going to execute rollback on graph. 2022-04-26T06:46:42.876Z [qtp2085002312-12] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Going to execute rollback on graph. 2022-04-26T06:46:45.077Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:293) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:269) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createSubCategoriesDao(CategoriesImportManager.java:161) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.isCreatedNewSubCategory(CategoriesImportManager.java:102) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:84) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:46:45.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba4f734d-c11d-4709-9673-c152c552481a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:46:45.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba4f734d-c11d-4709-9673-c152c552481a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8723abe 2022-04-26T06:46:45.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba4f734d-c11d-4709-9673-c152c552481a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:46:45.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba4f734d-c11d-4709-9673-c152c552481a PartnerName=SDC.BE The cassandra down nodes number is 0 2022-04-26T06:46:47.133Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:293) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:269) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createSubCategoriesDao(CategoriesImportManager.java:161) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.isCreatedNewSubCategory(CategoriesImportManager.java:102) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:84) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:46:48.615Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ba4f734d-c11d-4709-9673-c152c552481a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:46:48.615Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ba4f734d-c11d-4709-9673-c152c552481a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:46:49.240Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:293) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:269) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createSubCategoriesDao(CategoriesImportManager.java:161) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.isCreatedNewSubCategory(CategoriesImportManager.java:102) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:84) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:46:50.286Z [cluster1-nio-worker-1] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.233.72.7:9042 com.datastax.driver.core.exceptions.TransportException: [cassandra.onap/10.233.72.7:9042] Connection has been closed at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:821) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:46:50.287Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.233.72.7:9042 com.datastax.driver.core.exceptions.ConnectionException: [cassandra.onap/10.233.72.7:9042] Pool was closed during initialization at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:173) at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:158) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1080) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.google.common.util.concurrent.CollectionFuture.handleAllCompleted(CollectionFuture.java:69) at com.google.common.util.concurrent.AggregateFuture.processCompleted(AggregateFuture.java:282) at com.google.common.util.concurrent.AggregateFuture.decrementCountAndMaybeComplete(AggregateFuture.java:264) at com.google.common.util.concurrent.AggregateFuture.access$200(AggregateFuture.java:42) at com.google.common.util.concurrent.AggregateFuture$1.run(AggregateFuture.java:147) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:100) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.datastax.driver.core.Connection$Future.onException(Connection.java:1518) at com.datastax.driver.core.Connection$Dispatcher.errorOutAllHandler(Connection.java:1340) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:821) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2022-04-26T06:46:50.287Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5c340b9-1362-45dd-9359-dd766806a310 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:46:50.287Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5c340b9-1362-45dd-9359-dd766806a310 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@179befe4 2022-04-26T06:46:50.288Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5c340b9-1362-45dd-9359-dd766806a310 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:46:50.288Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5c340b9-1362-45dd-9359-dd766806a310 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:46:50.368Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:46:50.383Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:46:51.788Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:293) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:269) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createSubCategoriesDao(CategoriesImportManager.java:161) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.isCreatedNewSubCategory(CategoriesImportManager.java:102) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:84) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:46:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:46:52.687Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:46:52.687Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:46:53.326Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=d5c340b9-1362-45dd-9359-dd766806a310 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:46:53.327Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=d5c340b9-1362-45dd-9359-dd766806a310 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:46:53.893Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:293) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:269) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createSubCategoriesDao(CategoriesImportManager.java:161) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.isCreatedNewSubCategory(CategoriesImportManager.java:102) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:84) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:46:54.427Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=591b2564-88c0-40d2-9589-7b30492835e3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:46:54.428Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=591b2564-88c0-40d2-9589-7b30492835e3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@487e0fb3 2022-04-26T06:46:54.428Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=591b2564-88c0-40d2-9589-7b30492835e3 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:46:54.428Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=591b2564-88c0-40d2-9589-7b30492835e3 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:46:56.115Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:293) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:269) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createSubCategoriesDao(CategoriesImportManager.java:161) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.isCreatedNewSubCategory(CategoriesImportManager.java:102) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:84) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:46:57.462Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=591b2564-88c0-40d2-9589-7b30492835e3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:46:57.463Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=591b2564-88c0-40d2-9589-7b30492835e3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:46:58.124Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e87611bb-8ac8-4539-a527-1ae7fe93f5cd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:46:58.124Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e87611bb-8ac8-4539-a527-1ae7fe93f5cd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5996d4c 2022-04-26T06:46:58.124Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e87611bb-8ac8-4539-a527-1ae7fe93f5cd PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:46:58.124Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e87611bb-8ac8-4539-a527-1ae7fe93f5cd PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:46:58.320Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:293) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:269) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createSubCategoriesDao(CategoriesImportManager.java:161) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.isCreatedNewSubCategory(CategoriesImportManager.java:102) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:84) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:47:00.741Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:293) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:269) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createSubCategoriesDao(CategoriesImportManager.java:161) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.isCreatedNewSubCategory(CategoriesImportManager.java:102) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:84) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:47:01.163Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e87611bb-8ac8-4539-a527-1ae7fe93f5cd ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:47:01.163Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e87611bb-8ac8-4539-a527-1ae7fe93f5cd ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:47:01.931Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5271eff7-1dae-4101-a8e3-2f7d9acb04da PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:01.932Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5271eff7-1dae-4101-a8e3-2f7d9acb04da PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6f7764cd 2022-04-26T06:47:01.932Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5271eff7-1dae-4101-a8e3-2f7d9acb04da PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:01.932Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5271eff7-1dae-4101-a8e3-2f7d9acb04da PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:03.545Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:293) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:269) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createSubCategoriesDao(CategoriesImportManager.java:161) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.isCreatedNewSubCategory(CategoriesImportManager.java:102) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:84) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:47:04.963Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=5271eff7-1dae-4101-a8e3-2f7d9acb04da ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:47:04.964Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=5271eff7-1dae-4101-a8e3-2f7d9acb04da ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:47:05.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:47:05.376Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:47:05.736Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa68ad76-7335-4583-b3a6-76b0efdedc02 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:05.736Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa68ad76-7335-4583-b3a6-76b0efdedc02 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@46e68aff 2022-04-26T06:47:05.736Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa68ad76-7335-4583-b3a6-76b0efdedc02 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:05.737Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa68ad76-7335-4583-b3a6-76b0efdedc02 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:06.352Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:293) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:269) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createSubCategoriesDao(CategoriesImportManager.java:161) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.isCreatedNewSubCategory(CategoriesImportManager.java:102) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:84) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:47:07.678Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:47:08.767Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=fa68ad76-7335-4583-b3a6-76b0efdedc02 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:47:08.768Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=fa68ad76-7335-4583-b3a6-76b0efdedc02 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:47:08.916Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=98415a62-1e54-412b-b1ba-41b6186178de PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:08.917Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=98415a62-1e54-412b-b1ba-41b6186178de PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@224f3741 2022-04-26T06:47:08.917Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=98415a62-1e54-412b-b1ba-41b6186178de PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:08.917Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=98415a62-1e54-412b-b1ba-41b6186178de PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:09.959Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:293) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:269) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createSubCategoriesDao(CategoriesImportManager.java:161) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.isCreatedNewSubCategory(CategoriesImportManager.java:102) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:84) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:47:11.948Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=98415a62-1e54-412b-b1ba-41b6186178de ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:47:11.948Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=98415a62-1e54-412b-b1ba-41b6186178de ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:47:12.127Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d0979cb3-59b2-4155-9153-615fc032b173 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:12.127Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d0979cb3-59b2-4155-9153-615fc032b173 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3294550 2022-04-26T06:47:12.127Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d0979cb3-59b2-4155-9153-615fc032b173 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:12.127Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d0979cb3-59b2-4155-9153-615fc032b173 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:13.564Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:293) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:269) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createSubCategoriesDao(CategoriesImportManager.java:161) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.isCreatedNewSubCategory(CategoriesImportManager.java:102) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:84) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency ALL (3 replica were required but only 2 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:47:15.162Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=d0979cb3-59b2-4155-9153-615fc032b173 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:47:15.163Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=d0979cb3-59b2-4155-9153-615fc032b173 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:47:15.322Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef6902ca-8caf-430b-9be3-d654d5fd1378 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:15.323Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef6902ca-8caf-430b-9be3-d654d5fd1378 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7d4a4f72 2022-04-26T06:47:15.323Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef6902ca-8caf-430b-9be3-d654d5fd1378 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:15.323Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef6902ca-8caf-430b-9be3-d654d5fd1378 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:16.774Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:293) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:269) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createSubCategoriesDao(CategoriesImportManager.java:161) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.isCreatedNewSubCategory(CategoriesImportManager.java:102) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:84) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:47:18.359Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ef6902ca-8caf-430b-9be3-d654d5fd1378 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2022-04-26T06:47:18.360Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ef6902ca-8caf-430b-9be3-d654d5fd1378 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:229) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:144) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:401) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2022-04-26T06:47:18.514Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61baaa00-fc94-4723-8994-6fa3d060096d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:18.515Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61baaa00-fc94-4723-8994-6fa3d060096d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1188a83e 2022-04-26T06:47:18.515Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61baaa00-fc94-4723-8994-6fa3d060096d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:18.515Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61baaa00-fc94-4723-8994-6fa3d060096d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:18.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1aebe4ac-0b58-42a6-95be-0f8437a54aa9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:18.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1aebe4ac-0b58-42a6-95be-0f8437a54aa9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3cfe2d0f 2022-04-26T06:47:18.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1aebe4ac-0b58-42a6-95be-0f8437a54aa9 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:18.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1aebe4ac-0b58-42a6-95be-0f8437a54aa9 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:19.016Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5bd68df9-70b4-48ef-9ee1-257e2f9bded2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:19.016Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5bd68df9-70b4-48ef-9ee1-257e2f9bded2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@775ab8ce 2022-04-26T06:47:19.016Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5bd68df9-70b4-48ef-9ee1-257e2f9bded2 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:19.016Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5bd68df9-70b4-48ef-9ee1-257e2f9bded2 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:19.228Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04cba1e6-a27d-41d4-914f-27004345aacd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:19.228Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04cba1e6-a27d-41d4-914f-27004345aacd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@23472743 2022-04-26T06:47:19.228Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04cba1e6-a27d-41d4-914f-27004345aacd PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:19.228Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04cba1e6-a27d-41d4-914f-27004345aacd PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:19.718Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=120a6a18-a076-4289-8a58-ba21f854879a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:19.719Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=120a6a18-a076-4289-8a58-ba21f854879a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@363cec54 2022-04-26T06:47:19.719Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=120a6a18-a076-4289-8a58-ba21f854879a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:19.719Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=120a6a18-a076-4289-8a58-ba21f854879a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:20.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:47:20.377Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:47:20.413Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5a9d761-15c9-45e7-811f-8256587d031f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:20.413Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5a9d761-15c9-45e7-811f-8256587d031f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7be739c9 2022-04-26T06:47:20.414Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5a9d761-15c9-45e7-811f-8256587d031f PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:20.414Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5a9d761-15c9-45e7-811f-8256587d031f PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:21.316Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ef1436f-7cd8-45c7-be81-fec58f4223ab PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:21.316Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ef1436f-7cd8-45c7-be81-fec58f4223ab PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@59d972b3 2022-04-26T06:47:21.317Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ef1436f-7cd8-45c7-be81-fec58f4223ab PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:21.317Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ef1436f-7cd8-45c7-be81-fec58f4223ab PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:22.315Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=faad61b8-0523-42cc-b3f1-5e67594bbc2d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:22.315Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=faad61b8-0523-42cc-b3f1-5e67594bbc2d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@433e3bf4 2022-04-26T06:47:22.316Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=faad61b8-0523-42cc-b3f1-5e67594bbc2d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:22.316Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=faad61b8-0523-42cc-b3f1-5e67594bbc2d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:23.219Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:293) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:269) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createSubCategoriesDao(CategoriesImportManager.java:161) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.isCreatedNewSubCategory(CategoriesImportManager.java:102) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:84) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:47:23.220Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d2ed32a-e3ea-4593-8c03-98ea9789c3da PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:23.220Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d2ed32a-e3ea-4593-8c03-98ea9789c3da PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1231318b 2022-04-26T06:47:23.220Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d2ed32a-e3ea-4593-8c03-98ea9789c3da PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:23.220Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d2ed32a-e3ea-4593-8c03-98ea9789c3da PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:24.314Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=486db574-ff27-4b08-8771-3f25ad2c5f10 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:24.314Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=486db574-ff27-4b08-8771-3f25ad2c5f10 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@50da7b75 2022-04-26T06:47:24.314Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=486db574-ff27-4b08-8771-3f25ad2c5f10 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:24.314Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=486db574-ff27-4b08-8771-3f25ad2c5f10 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:25.313Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37b19785-69b9-4011-bd2c-72f063fc459d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:25.314Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37b19785-69b9-4011-bd2c-72f063fc459d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4204d737 2022-04-26T06:47:25.314Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37b19785-69b9-4011-bd2c-72f063fc459d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:25.314Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37b19785-69b9-4011-bd2c-72f063fc459d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:26.313Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4569ed27-51f0-4b03-94e4-d2930555342e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:26.314Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4569ed27-51f0-4b03-94e4-d2930555342e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@70d7b518 2022-04-26T06:47:26.314Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4569ed27-51f0-4b03-94e4-d2930555342e PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:26.314Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4569ed27-51f0-4b03-94e4-d2930555342e PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:27.312Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41643f24-89b5-400b-ae6e-d24382124fb9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:27.312Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41643f24-89b5-400b-ae6e-d24382124fb9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6b717fe0 2022-04-26T06:47:27.312Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41643f24-89b5-400b-ae6e-d24382124fb9 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:27.312Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41643f24-89b5-400b-ae6e-d24382124fb9 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:28.226Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a4fa014-bc19-405b-bac9-dbd647dab86c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:28.227Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a4fa014-bc19-405b-bac9-dbd647dab86c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@68f3b028 2022-04-26T06:47:28.227Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a4fa014-bc19-405b-bac9-dbd647dab86c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:28.227Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a4fa014-bc19-405b-bac9-dbd647dab86c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:29.020Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9b1edb28-8604-4a78-bc5f-6c4420d3e4cc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:29.020Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9b1edb28-8604-4a78-bc5f-6c4420d3e4cc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@caa6f6e 2022-04-26T06:47:29.020Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9b1edb28-8604-4a78-bc5f-6c4420d3e4cc PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:29.020Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9b1edb28-8604-4a78-bc5f-6c4420d3e4cc PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:29.719Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:293) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:269) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createSubCategoriesDao(CategoriesImportManager.java:161) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.isCreatedNewSubCategory(CategoriesImportManager.java:102) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:84) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:47:30.325Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e787ea4b-2c38-448e-82c8-21f9c89bef45 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:30.326Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e787ea4b-2c38-448e-82c8-21f9c89bef45 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@41987496 2022-04-26T06:47:30.326Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e787ea4b-2c38-448e-82c8-21f9c89bef45 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:30.326Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e787ea4b-2c38-448e-82c8-21f9c89bef45 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:31.529Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1994f886-c122-4e73-9cd9-0d6319bfa050 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:31.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1994f886-c122-4e73-9cd9-0d6319bfa050 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@21769d3a 2022-04-26T06:47:31.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1994f886-c122-4e73-9cd9-0d6319bfa050 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:31.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1994f886-c122-4e73-9cd9-0d6319bfa050 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:33.013Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d610d447-a418-4dc4-bb81-72e837413b0a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:33.013Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d610d447-a418-4dc4-bb81-72e837413b0a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@54f10bcc 2022-04-26T06:47:33.013Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d610d447-a418-4dc4-bb81-72e837413b0a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:33.014Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d610d447-a418-4dc4-bb81-72e837413b0a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:34.034Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76310c80-e699-4126-a873-f25f794b253d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:34.034Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76310c80-e699-4126-a873-f25f794b253d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@bffcf23 2022-04-26T06:47:34.034Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76310c80-e699-4126-a873-f25f794b253d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:34.034Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76310c80-e699-4126-a873-f25f794b253d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:34.834Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=baee7e5d-c36c-41ea-abb2-b8d37d13db96 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:34.835Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=baee7e5d-c36c-41ea-abb2-b8d37d13db96 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9c1260e 2022-04-26T06:47:34.835Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=baee7e5d-c36c-41ea-abb2-b8d37d13db96 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:34.835Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=baee7e5d-c36c-41ea-abb2-b8d37d13db96 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:35.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:47:35.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:47:35.940Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cefb87a7-f268-48d7-a47d-cade0c978f08 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:35.941Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cefb87a7-f268-48d7-a47d-cade0c978f08 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1c4be541 2022-04-26T06:47:35.941Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cefb87a7-f268-48d7-a47d-cade0c978f08 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:35.941Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cefb87a7-f268-48d7-a47d-cade0c978f08 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:37.133Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cce207d6-14ca-44a7-bafb-162f179b2bc3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:37.133Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cce207d6-14ca-44a7-bafb-162f179b2bc3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4038326f 2022-04-26T06:47:37.134Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cce207d6-14ca-44a7-bafb-162f179b2bc3 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:37.134Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cce207d6-14ca-44a7-bafb-162f179b2bc3 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:37.941Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29692f9d-255e-4af0-a127-5e678fc77313 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:37.942Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29692f9d-255e-4af0-a127-5e678fc77313 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@35cb6b9c 2022-04-26T06:47:37.942Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29692f9d-255e-4af0-a127-5e678fc77313 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:37.943Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29692f9d-255e-4af0-a127-5e678fc77313 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:38.736Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e49aeed-01b3-4f47-b217-da8ef5e68537 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:38.737Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e49aeed-01b3-4f47-b217-da8ef5e68537 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@35baca7d 2022-04-26T06:47:38.737Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e49aeed-01b3-4f47-b217-da8ef5e68537 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:38.737Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e49aeed-01b3-4f47-b217-da8ef5e68537 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:39.427Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee30ea67-7d52-46a2-90ed-b19a2174e42a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:39.427Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee30ea67-7d52-46a2-90ed-b19a2174e42a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@76185066 2022-04-26T06:47:39.428Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee30ea67-7d52-46a2-90ed-b19a2174e42a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:39.428Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee30ea67-7d52-46a2-90ed-b19a2174e42a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:39.844Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7f1c946-c884-40bf-84d1-a0ae0eb4f39d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:39.844Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7f1c946-c884-40bf-84d1-a0ae0eb4f39d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7732bf80 2022-04-26T06:47:39.844Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7f1c946-c884-40bf-84d1-a0ae0eb4f39d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:39.844Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7f1c946-c884-40bf-84d1-a0ae0eb4f39d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:40.038Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=17017e3e-cb74-42e4-81a4-d1b39ededc23 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:40.038Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=17017e3e-cb74-42e4-81a4-d1b39ededc23 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@19cbbd71 2022-04-26T06:47:40.038Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=17017e3e-cb74-42e4-81a4-d1b39ededc23 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:40.038Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=17017e3e-cb74-42e4-81a4-d1b39ededc23 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:40.236Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=190d4cdb-58ef-4a62-bf4c-5db3d0a9ba99 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:40.236Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=190d4cdb-58ef-4a62-bf4c-5db3d0a9ba99 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7a416517 2022-04-26T06:47:40.237Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=190d4cdb-58ef-4a62-bf4c-5db3d0a9ba99 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:40.237Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=190d4cdb-58ef-4a62-bf4c-5db3d0a9ba99 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:40.437Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8e5d50c-4ab1-4f29-8fd2-b874408b4978 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:40.437Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8e5d50c-4ab1-4f29-8fd2-b874408b4978 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4e6e8159 2022-04-26T06:47:40.437Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8e5d50c-4ab1-4f29-8fd2-b874408b4978 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:40.437Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8e5d50c-4ab1-4f29-8fd2-b874408b4978 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:40.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d00468a1-9a77-4fd7-b52b-573550a00772 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:40.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d00468a1-9a77-4fd7-b52b-573550a00772 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2cc2f9ab 2022-04-26T06:47:40.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d00468a1-9a77-4fd7-b52b-573550a00772 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:40.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d00468a1-9a77-4fd7-b52b-573550a00772 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:40.831Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02572ac1-eb06-4993-a080-6ff44ee75515 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:40.831Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02572ac1-eb06-4993-a080-6ff44ee75515 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6826e977 2022-04-26T06:47:40.831Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02572ac1-eb06-4993-a080-6ff44ee75515 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:40.831Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02572ac1-eb06-4993-a080-6ff44ee75515 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:41.021Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26ebbf5e-b66a-435d-afb8-387780a91c8d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:41.021Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26ebbf5e-b66a-435d-afb8-387780a91c8d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@19be40b9 2022-04-26T06:47:41.021Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26ebbf5e-b66a-435d-afb8-387780a91c8d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:41.021Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26ebbf5e-b66a-435d-afb8-387780a91c8d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:41.225Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a22fa0e0-84c2-4239-a336-ff4ac67e0dbb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:41.225Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a22fa0e0-84c2-4239-a336-ff4ac67e0dbb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@23e06e9e 2022-04-26T06:47:41.225Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a22fa0e0-84c2-4239-a336-ff4ac67e0dbb PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:41.225Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a22fa0e0-84c2-4239-a336-ff4ac67e0dbb PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:41.383Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d100a536-2a2d-4198-9f43-fef6fbec0729 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:41.384Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d100a536-2a2d-4198-9f43-fef6fbec0729 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@18d4c4e8 2022-04-26T06:47:41.384Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d100a536-2a2d-4198-9f43-fef6fbec0729 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:41.384Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d100a536-2a2d-4198-9f43-fef6fbec0729 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:41.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=99c7d7a8-71f5-4289-ab65-2a648621d567 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:41.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=99c7d7a8-71f5-4289-ab65-2a648621d567 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4766fbd4 2022-04-26T06:47:41.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=99c7d7a8-71f5-4289-ab65-2a648621d567 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:41.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=99c7d7a8-71f5-4289-ab65-2a648621d567 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:41.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a6317d5f-2e00-4074-9f78-2d5452188058 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:41.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a6317d5f-2e00-4074-9f78-2d5452188058 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2c36d009 2022-04-26T06:47:41.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a6317d5f-2e00-4074-9f78-2d5452188058 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:41.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a6317d5f-2e00-4074-9f78-2d5452188058 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:41.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21f620be-451c-426d-b131-912fdaf50b83 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:41.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21f620be-451c-426d-b131-912fdaf50b83 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@c82df8f 2022-04-26T06:47:41.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21f620be-451c-426d-b131-912fdaf50b83 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:41.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21f620be-451c-426d-b131-912fdaf50b83 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:42.148Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60f6dee2-f86c-40a6-b0b3-c3f0a824ca9d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:42.149Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60f6dee2-f86c-40a6-b0b3-c3f0a824ca9d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@32cfa973 2022-04-26T06:47:42.149Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60f6dee2-f86c-40a6-b0b3-c3f0a824ca9d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:42.149Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60f6dee2-f86c-40a6-b0b3-c3f0a824ca9d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:42.529Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:293) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:269) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createSubCategoriesDao(CategoriesImportManager.java:161) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.isCreatedNewSubCategory(CategoriesImportManager.java:102) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:84) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:47:42.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ccd108e-35c9-41cd-bcc5-4c0e005a435c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:42.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ccd108e-35c9-41cd-bcc5-4c0e005a435c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@42875ca5 2022-04-26T06:47:42.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ccd108e-35c9-41cd-bcc5-4c0e005a435c PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:42.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ccd108e-35c9-41cd-bcc5-4c0e005a435c PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:45.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=42db4fb9-25f7-40da-a24c-8337043e8990 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:45.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=42db4fb9-25f7-40da-a24c-8337043e8990 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@58ab7a64 2022-04-26T06:47:45.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=42db4fb9-25f7-40da-a24c-8337043e8990 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:45.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=42db4fb9-25f7-40da-a24c-8337043e8990 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:48.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b7589e8-dd16-4c82-87d8-4d963596215f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:48.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b7589e8-dd16-4c82-87d8-4d963596215f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@42cfb859 2022-04-26T06:47:48.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b7589e8-dd16-4c82-87d8-4d963596215f PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:48.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b7589e8-dd16-4c82-87d8-4d963596215f PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:50.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:47:50.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:47:52.425Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=193fb614-80a2-4a40-a74f-2812322d2415 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:52.426Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=193fb614-80a2-4a40-a74f-2812322d2415 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@d841542 2022-04-26T06:47:52.426Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=193fb614-80a2-4a40-a74f-2812322d2415 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:52.426Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=193fb614-80a2-4a40-a74f-2812322d2415 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:52.685Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2022-04-26T06:47:52.686Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2022-04-26T06:47:52.686Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as jsVxexVZMKFmNLbu) ... 2022-04-26T06:47:55.317Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f12c8e0-d02a-442a-9ab1-3acf61e57124 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:55.317Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f12c8e0-d02a-442a-9ab1-3acf61e57124 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@49d74d4b 2022-04-26T06:47:55.317Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f12c8e0-d02a-442a-9ab1-3acf61e57124 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:55.318Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f12c8e0-d02a-442a-9ab1-3acf61e57124 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:47:55.416Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:293) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:269) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createSubCategoriesDao(CategoriesImportManager.java:161) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.isCreatedNewSubCategory(CategoriesImportManager.java:102) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:84) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:47:58.516Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=716fd6a8-fc4a-4a71-9bcb-c1a7ff3ff000 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:47:58.516Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=716fd6a8-fc4a-4a71-9bcb-c1a7ff3ff000 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@64d46f36 2022-04-26T06:47:58.516Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=716fd6a8-fc4a-4a71-9bcb-c1a7ff3ff000 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:47:58.517Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=716fd6a8-fc4a-4a71-9bcb-c1a7ff3ff000 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:48:01.431Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=241022ca-fd26-498c-96cd-9b446808a121 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:48:01.431Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=241022ca-fd26-498c-96cd-9b446808a121 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@61def695 2022-04-26T06:48:01.431Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=241022ca-fd26-498c-96cd-9b446808a121 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:48:01.431Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=241022ca-fd26-498c-96cd-9b446808a121 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:48:04.328Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9bd64901-41bf-46cc-a6c5-01c2125b9fc8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:48:04.329Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9bd64901-41bf-46cc-a6c5-01c2125b9fc8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3e11a45e 2022-04-26T06:48:04.329Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9bd64901-41bf-46cc-a6c5-01c2125b9fc8 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:48:04.329Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9bd64901-41bf-46cc-a6c5-01c2125b9fc8 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:48:05.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:48:05.377Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:48:07.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6aeabedc-a44d-462d-88d3-6b2ebab980d6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:48:07.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6aeabedc-a44d-462d-88d3-6b2ebab980d6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4d5f0bc2 2022-04-26T06:48:07.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6aeabedc-a44d-462d-88d3-6b2ebab980d6 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:48:07.435Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6aeabedc-a44d-462d-88d3-6b2ebab980d6 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:48:07.672Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:48:09.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c969d88-12a9-438e-af51-d026f3c6f2ec PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:48:09.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c969d88-12a9-438e-af51-d026f3c6f2ec PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@a356080 2022-04-26T06:48:09.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c969d88-12a9-438e-af51-d026f3c6f2ec PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:48:09.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c969d88-12a9-438e-af51-d026f3c6f2ec PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:48:12.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a892d89d-59a6-4019-8fd1-5b98793d0057 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:48:12.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a892d89d-59a6-4019-8fd1-5b98793d0057 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1c49550d 2022-04-26T06:48:12.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a892d89d-59a6-4019-8fd1-5b98793d0057 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:48:12.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a892d89d-59a6-4019-8fd1-5b98793d0057 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:48:15.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48008bbc-0283-4d30-bd45-f0ed63156050 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:48:15.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48008bbc-0283-4d30-bd45-f0ed63156050 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2fc9318e 2022-04-26T06:48:15.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48008bbc-0283-4d30-bd45-f0ed63156050 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:48:15.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48008bbc-0283-4d30-bd45-f0ed63156050 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:48:18.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=092a0dca-cc9e-4745-bada-2abfe5f25b97 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:48:18.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=092a0dca-cc9e-4745-bada-2abfe5f25b97 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1afd3eae 2022-04-26T06:48:18.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=092a0dca-cc9e-4745-bada-2abfe5f25b97 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:48:18.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=092a0dca-cc9e-4745-bada-2abfe5f25b97 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:48:20.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:48:20.375Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:48:21.023Z [qtp2085002312-12] ERROR o.j.g.database.StandardJanusGraph RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Could not commit transaction [450] due to storage exception in commit org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:293) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:269) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createSubCategoriesDao(CategoriesImportManager.java:161) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.isCreatedNewSubCategory(CategoriesImportManager.java:102) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:84) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT1M40S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 122 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 123 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:48:21.024Z [qtp2085002312-12] ERROR o.j.g.database.StandardJanusGraph RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Could not commit transaction [450] due to exception org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:293) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateSubCategory(ElementOperation.java:269) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createSubCategoriesDao(CategoriesImportManager.java:161) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.isCreatedNewSubCategory(CategoriesImportManager.java:102) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:84) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT1M40S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 122 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 123 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:48:21.029Z [qtp2085002312-12] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Going to execute rollback on graph. 2022-04-26T06:48:21.114Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:199) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:176) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesDao(CategoriesImportManager.java:179) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:75) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:48:21.215Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:199) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:176) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesDao(CategoriesImportManager.java:179) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:75) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:48:21.314Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:199) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:176) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesDao(CategoriesImportManager.java:179) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:75) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:48:21.414Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:199) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:176) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesDao(CategoriesImportManager.java:179) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:75) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:48:21.513Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:199) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:176) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesDao(CategoriesImportManager.java:179) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:75) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:48:21.614Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:199) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:176) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesDao(CategoriesImportManager.java:179) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:75) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:48:21.722Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:199) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:176) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesDao(CategoriesImportManager.java:179) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:75) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:48:21.915Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:199) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:176) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesDao(CategoriesImportManager.java:179) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:75) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:48:22.114Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:199) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:176) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesDao(CategoriesImportManager.java:179) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:75) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:48:22.227Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:199) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:176) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesDao(CategoriesImportManager.java:179) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:75) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:48:22.515Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:199) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:176) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesDao(CategoriesImportManager.java:179) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:75) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:48:22.535Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01292969-5701-474e-a770-19d792c57d0d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:48:22.535Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01292969-5701-474e-a770-19d792c57d0d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3b6d7c11 2022-04-26T06:48:22.535Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01292969-5701-474e-a770-19d792c57d0d PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:48:22.535Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01292969-5701-474e-a770-19d792c57d0d PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:48:22.722Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:199) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:176) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesDao(CategoriesImportManager.java:179) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:75) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:48:23.215Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:199) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:176) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesDao(CategoriesImportManager.java:179) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:75) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:48:23.623Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:199) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:176) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesDao(CategoriesImportManager.java:179) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:75) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:48:24.428Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:199) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:176) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesDao(CategoriesImportManager.java:179) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:75) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:48:25.313Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:199) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:176) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesDao(CategoriesImportManager.java:179) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:75) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:48:25.424Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84eac2bf-c7df-4456-ada9-c9fb89c47a7a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:48:25.425Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84eac2bf-c7df-4456-ada9-c9fb89c47a7a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6ddaa74c 2022-04-26T06:48:25.425Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84eac2bf-c7df-4456-ada9-c9fb89c47a7a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:48:25.425Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84eac2bf-c7df-4456-ada9-c9fb89c47a7a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:48:26.120Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:199) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:176) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesDao(CategoriesImportManager.java:179) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:75) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:48:27.726Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:199) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:176) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesDao(CategoriesImportManager.java:179) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:75) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:48:28.419Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d38e34a8-c8c7-4aaa-b8df-9c4b5ecbda6a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:48:28.419Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d38e34a8-c8c7-4aaa-b8df-9c4b5ecbda6a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@b79ae38 2022-04-26T06:48:28.419Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d38e34a8-c8c7-4aaa-b8df-9c4b5ecbda6a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:48:28.419Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d38e34a8-c8c7-4aaa-b8df-9c4b5ecbda6a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:48:30.937Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:199) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:176) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesDao(CategoriesImportManager.java:179) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:75) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:48:31.428Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a66e1366-13d0-4a3e-bc3f-c5a03c81472a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:48:31.428Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a66e1366-13d0-4a3e-bc3f-c5a03c81472a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7802f9b7 2022-04-26T06:48:31.428Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a66e1366-13d0-4a3e-bc3f-c5a03c81472a PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:48:31.428Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a66e1366-13d0-4a3e-bc3f-c5a03c81472a PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:48:34.235Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:199) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:176) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesDao(CategoriesImportManager.java:179) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:75) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:48:34.437Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=addbc6b5-a56c-4f1c-8451-cf632a27e8f8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:48:34.437Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=addbc6b5-a56c-4f1c-8451-cf632a27e8f8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@28df9f52 2022-04-26T06:48:34.438Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=addbc6b5-a56c-4f1c-8451-cf632a27e8f8 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:48:34.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=addbc6b5-a56c-4f1c-8451-cf632a27e8f8 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:48:35.366Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/jsVxexVZMKFmNLbu (anonymous) ... 2022-04-26T06:48:35.382Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2022-04-26T06:48:37.340Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c13d669b-0ca6-44ee-9486-b6a7808fe6eb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:48:37.341Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c13d669b-0ca6-44ee-9486-b6a7808fe6eb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1087adc2 2022-04-26T06:48:37.341Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c13d669b-0ca6-44ee-9486-b6a7808fe6eb PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:48:37.341Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c13d669b-0ca6-44ee-9486-b6a7808fe6eb PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:48:37.444Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:199) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:176) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesDao(CategoriesImportManager.java:179) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:75) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:527) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2022-04-26T06:48:39.813Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f32fb46-ef50-4b26-a11b-2e8e1867a232 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:48:39.813Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f32fb46-ef50-4b26-a11b-2e8e1867a232 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@56c9c984 2022-04-26T06:48:39.813Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f32fb46-ef50-4b26-a11b-2e8e1867a232 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:48:39.813Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f32fb46-ef50-4b26-a11b-2e8e1867a232 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:48:40.650Z [qtp2085002312-12] INFO o.j.d.util.BackendOperation RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=onap-sdc-be-6b55bdfcb4-c9qbb TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.233.65.41 ServiceName=/v1/catalog/uploadType/categories TargetServiceName=null EntryTimestamp=2022-04-26T06:46:32.011Z InvocationID=313a961b-5ef1-4cbf-91e2-6f580015ed3e ErrorCode=500 Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:124) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:79) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:740) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:299) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.commit(JanusGraphGenericDao.java:80) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:199) at org.openecomp.sdc.be.model.operations.impl.ElementOperation.updateCategory(ElementOperation.java:176) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesDao(CategoriesImportManager.java:179) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategoriesByDao(CategoriesImportManager.java:75) at org.openecomp.sdc.be.components.impl.CategoriesImportManager.createCategories(CategoriesImportManager.java:60) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$6(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.createElementsType(TypesUploadServlet.java:365) at org.openecomp.sdc.be.servlets.TypesUploadServlet.lambda$uploadCategories$7(TypesUploadServlet.java:215) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadElementTypeServletLogic(TypesUploadServlet.java:292) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories_aroundBody8(TypesUploadServlet.java:216) at org.openecomp.sdc.be.servlets.TypesUploadServlet$AjcClosure9.run(TypesUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.TypesUploadServlet.uploadCategories(TypesUploadServlet.java:214) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$FastClassBySpringCGLIB$$893260aa.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:58) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.openecomp.sdc.be.servlets.TypesUploadServlet$$EnhancerBySpringCGLIB$$2ce85f84.uploadCategories() at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:230) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.exceptions.UnavailableException.copy(UnavailableException.java:141) at com.datastax.driver.core.Responses$Error.asException(Responses.java:159) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (3 required but only 2 alive) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:78) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2022-04-26T06:48:42.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11ed5000-31dd-4a6d-a142-fc3413418e30 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:48:42.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11ed5000-31dd-4a6d-a142-fc3413418e30 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@39e94dab 2022-04-26T06:48:42.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11ed5000-31dd-4a6d-a142-fc3413418e30 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:48:42.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11ed5000-31dd-4a6d-a142-fc3413418e30 PartnerName=SDC.BE The cassandra down nodes number is 1 2022-04-26T06:48:45.813Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b455aa71-2d23-49b3-9ca6-de23fab200d8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2022-04-26T06:48:45.813Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b455aa71-2d23-49b3-9ca6-de23fab200d8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6d9870cb 2022-04-26T06:48:45.813Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b455aa71-2d23-49b3-9ca6-de23fab200d8 PartnerName=SDC.BE The number of cassandra nodes is:3 2022-04-26T06:48:45.814Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b455aa71-2d23-49b3-9ca6-de23fab200d8 PartnerName=SDC.BE The cassandra down nodes number is 1